CN112381173B - Image recognition-based mechanical arm multitask autonomous learning control method and system - Google Patents

Image recognition-based mechanical arm multitask autonomous learning control method and system Download PDF

Info

Publication number
CN112381173B
CN112381173B CN202011373252.0A CN202011373252A CN112381173B CN 112381173 B CN112381173 B CN 112381173B CN 202011373252 A CN202011373252 A CN 202011373252A CN 112381173 B CN112381173 B CN 112381173B
Authority
CN
China
Prior art keywords
mechanical arm
image
neural network
pooling
convolution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011373252.0A
Other languages
Chinese (zh)
Other versions
CN112381173A (en
Inventor
王敏
曾宇鹏
黄盛钊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN202011373252.0A priority Critical patent/CN112381173B/en
Publication of CN112381173A publication Critical patent/CN112381173A/en
Application granted granted Critical
Publication of CN112381173B publication Critical patent/CN112381173B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Manipulator (AREA)

Abstract

The invention discloses a method and a system for controlling multitask autonomous learning of a mechanical arm based on image recognition, wherein the method comprises the following steps: in the identification stage, a distributed collaborative learning control method of a group of isomorphic rigid mechanical arms under joint space joint tracks is developed, a communication topology is constructed to realize online experience sharing of neural network weights among the multiple mechanical arms, a constant neural network controller is designed by using the experience weights, a width learning system is constructed to be used as an image classifier, and a mapping table is established to establish one-to-one correspondence relationship between the image categories and the tracks of control tasks; in the identification stage, different control tasks of the mechanical arm are identified by using the constructed image classifier, and the constant neural network controller is called to realize the multitask autonomous control of the rigid mechanical arm. The invention can not only obtain and call experience knowledge from complex work tasks, but also monitor external work tasks in real time through the visual sensor, and realize intelligent autonomous control of the mechanical arm under various work tasks.

Description

Image recognition-based mechanical arm multitask autonomous learning control method and system
Technical Field
The invention relates to the technical field of robot arm control, in particular to a robot arm multitask autonomous learning control method and system based on image recognition.
Background
With the continuous expansion of the field of the application of the industrial 4.0 and the mechanical arm, people have higher and higher requirements on the intelligent degree of the mechanical arm, and not only the robot is expected to execute distributed complex tasks in a cooperative manner and learn, store and utilize experience knowledge from the work tasks so as to improve the work efficiency and the work quality of the robot, but also the robot is expected to realize the capability of autonomous control according to the current work task. However, the existing robot control technology can only realize high-performance control of a single work task, and cannot adopt one control technology to realize high-precision autonomous control of a plurality of work tasks.
In recent years, the theory of distributed collaborative learning and breadth learning develops rapidly. The collaborative learning can solve the problem that a group of robots with the same nonlinear uncertain dynamics utilize undirected topological communication among the robots under different respective reference tracks to acquire, express and apply experience knowledge of the nonlinear uncertain dynamics, effectively improves the information interaction capacity of the multiple robots on respective work tasks, and enables each robot to have the capacity of completing the work tasks of the robots in the group. Meanwhile, the mechanical arm is increasingly rich along with application scenes, and the trend of combining mode identification with application is increasingly obvious, so that the intelligent level of the mechanical arm is further improved. In the field of machine vision, in order to improve the accuracy of function approximation and mode classification, a network structure is continuously expanded towards the depth direction, so that the problems of long network training time, low speed, high hardware performance requirement and the like are brought. In order to improve the speed of network training and pattern recognition, a width learning theory expands neurons comprising feature mapping nodes and enhanced nodes in a width mode, and then calculates a network model of output weight through a ridge regression algorithm, so that an efficient learning framework of machine learning and pattern recognition is provided, and the problems of long training time and low speed caused by a deep network structure are effectively solved. The distributed collaborative learning and the breadth learning theory respectively have outstanding advantages in the fields of multitask control and pattern recognition. The distributed collaborative learning can provide a controller design capable of completing multiple tasks for the mechanical arm control system, the width learning system can classify external images, the combination of the two learning technologies can realize real-time monitoring and switching of work tasks of the mechanical arm in the task execution process, control performance is guaranteed, and multi-task autonomous intelligent control of the mechanical arm is realized.
Disclosure of Invention
Aiming at different control tasks, the invention designs a constant neural network controller capable of completing different control tasks based on distributed collaborative learning; a width learning system based on a convolutional neural network is constructed as an image classifier by identifying a control task track, a mapping table is established to establish a one-to-one correspondence relationship between the image category and the control task track, so that the change of an external image can be responded by a constant neural network controller, and when a mechanical arm detects that a work task changes in the process of executing the control task, the constant neural network controller is called to realize the multi-task autonomous control of the mechanical arm.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a mechanical arm multitask autonomous learning control method based on image recognition, which comprises the following steps of:
constructing a kinematic model, a dynamic model and a plurality of expected regression Cartesian space trajectory models of a group of isomorphic mechanical arms in an identification stage, and defining each expected regression trajectory as a work task;
in the identification stage, mapping Cartesian space trajectories into joint space regression trajectories of the mechanical arm according to the constructed inverse kinematics model of the mechanical arm, realizing online experience sharing of weights of a group of isomorphic mechanical arm neural networks under joint space joint trajectories by constructing a communication topological structure, and designing an adaptive neural network collaborative learning controller as follows:
Figure GDA0003528357290000021
Figure GDA0003528357290000022
wherein, ci,2The gain matrix is controlled for the designed positive constants,
Figure GDA0003528357290000031
is an estimate of the ideal neural network weight, S (psi)i) Is a vector psiiIs a Gaussian radial basis function of the input, aijIs the adjacent of the communication topology between the mechanical armsTo the matrix element, zx,pBeing error variable, ΓiTo design a positive definite diagonal matrix, σiIs a constant; rho is a cooperative gain coefficient;
time period [ t ] for consistent convergence of closed-loop system after cooperative learning control1,t2]And selecting a constant neural network weight form as follows:
Figure GDA0003528357290000032
designing a constant neural network controller by using the empirical weight:
Figure GDA0003528357290000033
in the identification stage, by utilizing the feature extraction and generalization capability of a convolutional neural network, training images are subjected to feature extraction through convolutional pooling operation to obtain a feature layer, the feature layer is subjected to convolutional pooling operation to generalize the features to obtain an enhancement layer, the feature layer and the enhancement layer are fully linked to an output layer, and a class of convolutional neural network-based width learning system is constructed to serve as an image classifier;
in the identification stage, the image classification and a plurality of work tasks in the image classifier are set with a mapping table, and a one-to-one correspondence relationship is established and expressed as:
f:κ→Σy
where κ ∈ {1, 2.,. K } is the label of the image classification, ΣyAll work tasks which can be completed by the mechanical arm group;
in the identification stage, a camera monitors environmental changes in real time to obtain an image label;
switching to a working task corresponding to the image label through the established mapping table in the identification stage;
and the mechanical arm calls a constant neural network controller in the identification stage to complete the multi-task autonomous control of the mechanical arm.
As an optimal technical scheme, a group of kinematics models of isomorphic mechanical arms are constructed in an identification stage, and the method specifically comprises the following steps:
positive kinematics represents the mapping of the mechanical arm from joint space to cartesian space:
χ＝T(q)
inverse kinematics represents the mapping of the mechanical arm from cartesian space to joint space:
q＝InvT(χ)
wherein, χ ═ x, y, z]TFor the arm tip position in Cartesian space, q ═ θ1,θ2,...,θn]TFor the angular displacement of the mechanical arm in the joint space, n corresponds to the number of joints of the mechanical arm, T (-) is a positive kinematics mapping relation, and InvT (-) is an inverse kinematics mapping relation.
As a preferred technical solution, the dynamic model of the set of isomorphic mechanical arms is represented as:
Figure GDA0003528357290000041
wherein x isi,1＝[xi,1,1,xi,1,2,…,xi,1,n]T、xi,2＝[xi,2,1,xi,2,2,…,xi,2,n]TRespectively, the angular displacement and angular velocity of the joint of the mechanical arm, M (x)i,1) Is the inertia matrix of the arm, C (x)i,1,xi,2) As a centripetal force matrix, G (x)i,1) Is a gravity term, F (x)i,2) Is the friction force vector uiTo control the torque.
As a preferred technical solution, the cartesian space trajectory model is expressed as:
Figure GDA0003528357290000042
wherein the content of the first and second substances,
Figure GDA0003528357290000043
respectively the expected regression trajectory of the end of the robot arm in cartesian space under the work task k,
Figure GDA0003528357290000044
respectively, known as continuously-derivable periodic functions.
As a preferred technical solution, the mapping of the cartesian space trajectory into the mechanical arm joint space regression trajectory according to the constructed mechanical arm inverse kinematics model is specifically represented as:
qd＝InvT(χd)
wherein, χdRepresenting a Cartesian space trajectory, qdThe regression trajectory of the mechanical arm joint space is shown, and InvT (cndot) shows the inverse kinematics mapping relation.
As a preferred technical scheme, the training image is subjected to feature extraction through a convolution pooling operation to obtain a feature layer, and the specific steps include:
for j input image in training data setj＝Rw0×h0The following convolution and pooling operations were performed:
Figure GDA0003528357290000051
Ti p＝Pooling(Ti c,pf,spf)∈Rw2×h2
wherein, Ti cAnd Ti pRepresenting the output images of the convolution Conv (-) and Pooling Pooling (-) operations, respectively,
Figure GDA0003528357290000052
randomly generated for the ith channel with a size θf× θ f1,2, etaf，ηfIs the total number of channels of the convolution kernel,
Figure GDA0003528357290000053
for the bias term, s, corresponding to the ith channelcf、spfStride, p, selected for convolution and pooling operations, respectivelyfFor pooling windows, w1 × h1 and w2 × h2 perform convolution and pooling operations, respectivelyThe size of the output image;
will { T }i p|i＝1,2,...,ηfRemodeling is the number of characteristic nodes as
Figure GDA0003528357290000054
Feature vector of
Figure GDA0003528357290000055
Figure GDA0003528357290000056
The training samples are processed by convolution pooling operation to obtain a characteristic layer
Figure GDA0003528357290000057
ξ (-) is the tanh or sigmoid activation function, and L is the number of training samples.
As a preferred technical solution, the feature layer is further subjected to a convolution pooling operation, and the features are generalized to obtain an enhancement layer, and the specific steps include:
the output image of the convolution and pooling operation is represented as:
Figure GDA0003528357290000058
Figure GDA0003528357290000059
wherein the content of the first and second substances,
Figure GDA00035283572900000510
and
Figure GDA00035283572900000511
output images representing convolution and Pooling operations, respectively, Conv (-) representing a convolution operation, Pooling (-) representing a Pooling operation,
Figure GDA00035283572900000512
randomly generated size theta for ith channele× θ e1,2, etae，ηeIs the total number of channels of the convolution kernel,
Figure GDA00035283572900000513
for the bias term, s, corresponding to the ith channelce、speStride, p, selected for convolution and pooling operations, respectivelyeFor pooling windows, w3 × h3, w4 × h4 are the sizes of the output images for performing convolution and pooling operations, respectively;
will be provided with
Figure GDA0003528357290000062
Remodeling as a characteristic number of nodes
Figure GDA0003528357290000063
Feature vector of
Figure GDA0003528357290000064
Figure GDA0003528357290000065
Finally, the enhancement layer is obtained
Figure GDA0003528357290000066
Wherein, L is the number of training samples, and ξ (-) is a tanh or sigmoid activation function.
As a preferred technical scheme, the feature layer and the enhancement layer are all linked to the output layer, and a convolutional neural network-based width learning system is constructed as an image classifier, and the specific steps include:
combining the characteristic nodes and the enhanced nodes into A ═ Z | E ], and in a training stage, calculating a pseudo-inverse value of A by using a ridge regression algorithm:
Figure GDA0003528357290000061
wherein I is the following matrix ATA is a unit matrix with the same size, and lambda is a regular term coefficient in a ridge regression algorithm;
the weight of the output layer can be obtained from Y ═ AW:
W＝A+Y
wherein Y ∈ RL*KTo train the output matrix of the set, L is the number of samples and K is the number of image labels.
As a preferred technical scheme, the camera monitors environmental changes in real time to obtain an image tag, and the specific steps include:
the camera captures an indication image;
preprocessing a captured image, wherein the preprocessing comprises the steps of region selection, binarization and dimension adjustment;
and taking the preprocessed image as the input of the trained image classifier to obtain the label of the image.
The invention also provides a mechanical arm multitask autonomous learning control system based on image recognition, which comprises: the system comprises a model building module, a space trajectory mapping module, a controller building module, a training module, a mapping table setting module, an image information acquisition module, a work task switching module and a control signal output module;
the model construction module is used for constructing a kinematic model, a dynamic model and a plurality of expected regression Cartesian space trajectory models of a group of isomorphic mechanical arms in an identification stage, and each expected regression trajectory is defined as a work task;
the space track mapping module is used for mapping the Cartesian space track into a mechanical arm joint space regression track according to the constructed mechanical arm inverse kinematics model in the identification stage;
the controller construction module is used for realizing online experience sharing of weights of a group of isomorphic manipulator neural networks under joint space joint tracks by constructing a communication topological structure, and the adaptive neural network collaborative learning controller is designed as follows:
Figure GDA0003528357290000071
Figure GDA0003528357290000072
wherein, ci,2The gain matrix is controlled for the designed positive constants,
Figure GDA0003528357290000073
is an estimate of the ideal neural network weight, S (psi)i) Is a vector psiiIs a Gaussian radial basis function of the input, aijIs a contiguous matrix element of the inter-arm communication topology, zi,pBeing error variable, ΓiTo design a positive definite diagonal matrix, σiIs a constant; rho is a cooperative gain coefficient;
time period [ t ] for consistent convergence of closed-loop system after cooperative learning control1,t2]And selecting a constant neural network weight form as follows:
Figure GDA0003528357290000074
designing a constant neural network controller by using the empirical weight:
Figure GDA0003528357290000075
the training module is used for utilizing the feature extraction and generalization capability of the convolutional neural network in the identification stage, carrying out feature extraction on a training image through convolutional pooling operation to obtain a feature layer, carrying out convolutional pooling operation on the feature layer to generalize the features to obtain an enhancement layer, and fully linking the feature layer and the enhancement layer to an output layer to construct a width learning system based on the convolutional neural network as an image classifier;
the mapping table setting module is used for setting mapping tables for the image categories and the plurality of work tasks in the image classifier in the identification stage, and establishing a one-to-one correspondence relationship which is expressed as follows:
f:κ→Σy
where κ ∈ {1, 2.,. K } is the label of the image classification, ΣyAll work tasks which can be completed by the mechanical arm group;
the image information acquisition module monitors environmental changes in real time through a camera in an identification stage to obtain an image tag;
the work task switching module is used for switching to the work task corresponding to the image label through the established mapping table in the identification stage;
the control signal output module is used for outputting control signals, and the mechanical arm calls the constant neural network controller in the identification stage to complete the multi-task autonomous control of the mechanical arm.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) the invention constructs a convolutional neural network-based width learning system as an image classifier, greatly reduces the model training time compared with the existing deep neural network algorithm, and improves the image recognition accuracy compared with the width learning algorithm.
(2) The invention adopts a collaborative learning technical scheme, solves the problem of autonomous control of various working tasks of the mechanical arm in a complex working environment, and achieves the effect that the mechanical arm adopts a learning control scheme to realize accurate control of various working tasks.
(3) The mechanical arm disclosed by the invention not only can acquire and call experience knowledge from complex work tasks, but also can monitor external work tasks in real time through the visual sensor, so that the intelligent autonomous control of the mechanical arm under various work tasks is realized.
Drawings
Fig. 1 is a flowchart of a method for controlling multitask autonomous learning of a robot arm based on image recognition according to the embodiment 1;
FIG. 2 is a schematic structural diagram of the robot arm system of this embodiment 1;
fig. 3 is a network diagram of a topology for controlling communication in cooperative learning of 3 robot arms according to this embodiment 1;
FIG. 4 is a top view of the working interface of the embodiment 1;
fig. 5 is a schematic diagram of a trajectory of an end of a robot in a simulation scene according to this embodiment 1;
FIG. 6 is a frame diagram of an image classifier based on convolutional neural network and width learning in this embodiment 1;
fig. 7 is a graph illustrating convergence of the RBF neural network weight norm consistency for 3 arms according to this embodiment 1;
FIG. 8 is a graph showing the variation of tracking error of angular displacement of the mechanical arm joint in the embodiment 1;
fig. 9 is a graph showing the tracking change of angular displacement of the mechanical arm joint in the embodiment 1.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example 1
As shown in fig. 1, the present embodiment provides a method for controlling multitask autonomous learning of a mechanical arm based on image recognition, including the following steps:
s1: in the identification stage, a group of kinematics and dynamics models of isomorphic mechanical arms and a plurality of expected regression Cartesian space trajectory models are constructed, and each expected regression trajectory is defined as a work task;
as shown in FIG. 2, the robot arm of this embodiment employs a three-link robot arm, the length of each link being a1＝75mm，a2＝245mm，a1 Joint 1, Joint 2 and Joint 3 are joints 1,2 and 3 of the three-link mechanical arm in the embodiment of the invention, wherein the Joint is 240 mm;
let a point in Cartesian space be P (P)x,py,pz) Corresponding angular displacement of the joint q, and for simplifying writing, contract ci＝cosθi，si＝sinθi，cij＝cos(θi+θj)，sij＝sin(θi+θj) According to the geometrical relationship of the joints, the positive kinematic relationship is as follows:
px＝c1(a2c2+a3c23)
py＝s1(a2c2+a3c23)
pz＝a1+a2s2+a3s23
based on the positive kinematic relation, the inverse kinematic relation of the mechanical arm can be solved,
solving for joint 3 angular displacement theta3: according to the geometrical relationship of the mechanical arm, the following results are obtained:
Figure GDA0003528357290000101
Figure GDA0003528357290000102
therefore theta31＝Atan2(s3,c3)，θ32＝-Atan2(s3,c3)
Solving joint 2 angular displacement theta2: using joint angle theta3And a positive kinematic relationship, can be obtained
Figure GDA0003528357290000103
Figure GDA0003528357290000104
According to the nature of trigonometric function
Figure GDA0003528357290000105
When the temperature of the water is higher than the set temperature,
Figure GDA0003528357290000106
Figure GDA0003528357290000107
when in use
Figure GDA0003528357290000108
When the temperature of the water is higher than the set temperature,
Figure GDA0003528357290000109
Figure GDA0003528357290000111
solving for joint 1 angular displacement theta1: from the spatial geometrical relationships
θ11＝Atan(px,py)
θ12＝Atan(-px,-py)
Wherein Atan (DEG) is an improved arc tangent function, and the value range of the Atan (DEG) can meet the angle range theta solved by the inverse kinematics algorithm of the mechanical armg∈[-π,π],g＝1,2,3。
The 4 sets of inverse solutions of the end of the mechanical arm at the position P are respectively (theta)11,θ21,θ31)，(θ11,θ23,θ32)，(θ12,θ22,θ31)，(θ12,θ24,θ32) When the Cartesian space is converted into the joint space, one inverse solution q (theta) is selected from 4 inverse solutions according to the principle that the distance between the front joint angle and the rear joint angle is the nearest and the sum of the rotation angles is the minimum (theta)1,θ2,θ3) And completing the inverse kinematics mapping of the mechanical arm:
q＝InvT(P)
establishing a group of dynamic models comprising 3 isomorphic three-link mechanical arms:
Figure GDA0003528357290000112
wherein x isi,1＝[xi,1,1,xi,1,2,xi,1,3]T、xi2＝[xi,2,1,xi,2,2,xi,2,3]TRespectively, the angular displacement and angular velocity of the joint of the mechanical arm, M (x)i,1) Is an inertia matrix of the robot arm, Cm(xi,1,xi,2) As a centripetal force matrix, G (x)i,1) Is a gravity term, F (x)i,2) Is the friction force vector ui＝[ui,1,ui,2,ui,3]TRepresenting the control input to the system.
Three expected regression trajectory models are given in this example as:
Figure GDA0003528357290000113
wherein the content of the first and second substances,
Figure GDA0003528357290000114
respectively is an expected regression trajectory of the tail end of the mechanical arm under the work task k in a Cartesian space,
Figure GDA0003528357290000115
respectively, are known continuously derivable periodic functions. In the present embodiment, given that the periods of the three work task modes are the same, T is 8s, and the reference trajectory in one period is given as follows:
Figure GDA0003528357290000121
Figure GDA0003528357290000122
s2: an identification stage, based on the constructed inverse kinematics of the mechanical arm, mapping the Cartesian space trajectory to a joint space regression trajectory of the mechanical arm, namely
Figure GDA0003528357290000123
Based on distributed synergeticsThe control algorithm is learned, 3 isomorphic mechanical arms utilize the constructed communication topology, and the joint locus q in the joint space is [ q ═ q [ ]1,q2,q3]And then, realizing online experience sharing of the neural network weight to obtain the self-adaptive neural network collaborative learning controller:
Figure GDA0003528357290000124
Figure GDA0003528357290000125
wherein, ci,2The gain matrix is controlled for the designed positive constants,
Figure GDA0003528357290000126
is an estimate of the ideal neural network weight, S (psi)i) Is a vector
Figure GDA0003528357290000127
Is the input gaussian radial basis function; a isijThe adjacent matrix elements of the communication topology between the mechanical arms; z is a radical of formulai,p(p ═ 1,2) is the error variable for the design controller process; gamma-shapediA normal number diagonal matrix is designed; sigmaiA small constant designed to improve the robustness of the controller; ρ is a cooperative gain coefficient. In this embodiment, c1,1＝c2,1＝4，c1,2＝c2,2＝38，Γi＝7.5，ρ＝0.8，σi0.0004, as shown in fig. 3, in the adaptive neural network collaborative learning control training of this embodiment, 3 mechanical arms are used for training three kinds of work tasks, and according to the communication topology network of 3 mechanical arm collaborative learning control, it can be known that the adjacency matrix element a isijThe formed mechanical arm group adjacency matrix is as follows:
Figure GDA0003528357290000131
after cooperative learningTime period t of consistent convergence of closed loop system1,t2]And selecting a constant neural network weight form as follows:
Figure GDA0003528357290000132
designing a constant neural network controller by using the empirical weight:
Figure GDA0003528357290000133
s3: in the identification stage, the excellent feature extraction and generalization capability of the convolutional neural network is utilized, and the training image is subjected to feature extraction through convolutional pooling operation to obtain a feature layer:
for j input image in training data setj＝Rw0×h0Convolution and pooling to obtain:
Figure GDA0003528357290000134
Ti p＝Pooling(Ti c,pf,spf)∈Rw2×h2
wherein, Ti cAnd Ti pRepresenting the output images of the convolution Conv (-) and Pooling (-) operations, respectively,
Figure GDA0003528357290000135
randomly generated size theta for ith channelf× θ f1,2, etaf，ηfIs the total number of the convolution kernel channels,
Figure GDA0003528357290000136
offset term, s, corresponding to the ith channelcf、spfStride, p, selected for convolution and pooling operations, respectivelyfFor pooling Window, w1 × h1, w2 × h2 are the output images for performing convolution and pooling operations, respectivelyThe size of (d);
will { T }i p|i＝1,2,...,ηfRemodeling is a characteristic number of nodes
Figure GDA0003528357290000137
Feature vector of
Figure GDA0003528357290000138
Figure GDA0003528357290000139
Performing the above operation on the L training samples to obtain a feature layer
Figure GDA00035283572900001310
ξ (-) is a tanh or sigmoid activation function;
in the embodiment, in the verification of the effect of the width learning classifier based on the convolutional neural network, training and testing of the classifier are performed in the established trajectory data set, and the data set comprises three types of labels of 'circle', 'flower' and 'heart'. The training set comprises 50 track maps, the test set comprises 10 track maps, the size of each image is 32 multiplied by 32, the number eta of convolution kernel channels selected by a characteristic layer is 2, the size theta is 5, and the step length s of the convolution and pooling operationc＝1，spSince p is a 2 × 2 mean pooling window, it is computationally obvious that the sizes w1 × h1 and w2 × h2 of the convolution and pooling operation output images are 28 × 28 and 14 × 14, respectively, and the number of final feature nodes is 28 × 28 and 14 × 14
Figure GDA0003528357290000141
ξ (·) of this example selects the tanh activation function.
As shown in fig. 4 and 5, in the system operation time period, the vision sensor sequentially obtains an image of the shape "circle-flower-heart", and the mechanical arm traces out the corresponding shape according to the image observed by the vision sensor.
In step S3 of this embodiment, the feature layer is further subjected to a convolution pooling operation to generalize the feature to obtain an enhancement layer:
Figure GDA0003528357290000142
Figure GDA0003528357290000143
wherein the content of the first and second substances,
Figure GDA0003528357290000144
and
Figure GDA0003528357290000145
representing the output images of the convolution Conv (-) and Pooling Pooling (-) operations, respectively,
Figure GDA0003528357290000146
randomly generated size theta for ith channele× θ e1,2, etae，ηeIs the total number of channels of the convolution kernel,
Figure GDA0003528357290000147
for the bias term, s, corresponding to the ith channelce、speStride, p, selected for convolution and pooling operations, respectivelyeFor pooling windows, w3 × h3, w4 × h4 are the sizes of the output images for performing convolution and pooling operations, respectively;
will be provided with
Figure GDA0003528357290000148
Remodel to an enhanced node number of
Figure GDA0003528357290000149
Feature vector of
Figure GDA00035283572900001410
Figure GDA00035283572900001411
Performing the above operations on the L training samples to obtain an enhancement layer
Figure GDA00035283572900001412
ξ (-) is a tanh or sigmoid activation function;
in this embodiment, the number η of convolution kernel channels selected for enhancement is 10, the size θ is 3, and the step s of the convolution and pooling operationsc＝1，spSince p is a 2 × 2 mean pooling window, it is computationally obvious that the sizes w1 × h1 and w2 × h2 of the convolution and pooling operation output images are 12 × 12 and 6 × 6, respectively, and the number of final feature nodes is calculated as
Figure GDA0003528357290000152
ξ (-) of this example selects the tanh activation function.
In this embodiment, the feature layer and the enhancement layer are all linked to the output layer, as shown in fig. 6, the width learning system based on the convolutional neural network is constructed as an image classifier:
combining the characteristic nodes and the enhanced nodes into A ═ Z | E ], and in a training stage, calculating a pseudo-inverse value of A by using a ridge regression algorithm:
Figure GDA0003528357290000151
wherein I is the heel matrix ATA is a unit matrix with the same size, and lambda is a regular term coefficient in a ridge regression algorithm;
the weight of the output layer can be obtained from Y ═ AW:
Figure GDA0003528357290000153
wherein Y ∈ RL*KFor the output matrix of the training set, N is the number of samples and K is the number of image labels.
S4: establishing a one-to-one correspondence relationship between the categories of the images in the width learning system obtained in the step S3 and the plurality of joint space trajectory setting mapping tables in the step S2:
f:κ→Σy
where κ ∈ {1,2,3} is the label of the image classification, ΣyAll work tasks which can be completed by the mechanical arm;
s5: the camera captures an image indication in real time to obtain an image label k:
s51: the camera captures an indication image;
s52: preprocessing the captured image such as region selection, binarization, dimension adjustment and the like;
s53: the label of the image is obtained by inputting the preprocessed image to the width learning classifier trained in step S3.
S6: in the identification stage, the work task corresponding to the image label is found through the mapping table established in the step S4;
s7: and in the identification stage, the mechanical arm calls the constant neural network collaborative learning controller in the mode.
In the present embodiment, x1、x2Is x1(0)＝[0,1.1829,-2.3055]T，x2(0)＝[0,0,0]TThe number of nodes of the neural network is 12500, and the central points are uniformly distributed in [ -0.6,0.6 [)]×[0.5,0.8]×[-2.9,-1.7]×[-1,1]×[-0.2,0.2]×[-1,1]Width of [0.375,0.125,0.375,0.625,0.125, 0.625)]T。
In order to illustrate the effectiveness and rapidity of the invention using the width learning system constructed by convolution as an image classifier, the embodiment trains and tests the established trajectory data set, and performs 3 independent experiments respectively, and the accuracy and training time are shown in table 1 below.
Table 1 table of 3 experimental results of the width learning image classifier
Number of experiments Rate of accuracy/%) Training time/s
1 100 0.0331
2 100 0.0289
3 100 0.0317
Average 100 0.0312
As can be seen from the above table, in 3 independent classification experiments, the accuracy rate reaches 100%, and the training time is short, so that the convolutional neural network-based width learning system serves as an image classifier, meets the application requirements in practical engineering, and can achieve better classification capability under the condition of complex images.
As shown in fig. 7, a convergent curve with the same weight norm of the RBF neural network of 3 arms is obtained; as shown in fig. 8, a variation curve of the angular displacement tracking error of the joint of the mechanical arm is obtained, and the tracking errors of the three joints of the mechanical arm are all within ± 0.025 rads; as shown in fig. 9, a curve of change between the joint angular displacement and the expected angular displacement of the mechanical arm in the simulation scene is obtained, so that the tail end of the mechanical arm can draw a corresponding image according to the information of the visual sensor, and the capability of the mechanical arm to autonomously complete multiple work tasks is embodied.
Example 2
The embodiment provides a mechanical arm multitask autonomous learning control system based on image recognition, which comprises: the system comprises a model building module, a space trajectory mapping module, a controller building module, a training module, a mapping table setting module, an image information acquisition module, a work task switching module and a control signal output module;
in this embodiment, the model construction module is configured to construct a set of kinematic models and kinetic models of a homogeneous mechanical arm and a plurality of expected regression cartesian space trajectory models in an identification stage, and define each expected regression trajectory as a work task;
in this embodiment, the space trajectory mapping module is configured to map a cartesian space trajectory into a mechanical arm joint space regression trajectory according to the constructed mechanical arm inverse kinematics model in the identification stage;
in this embodiment, the controller building module is configured to implement online experience sharing of weights of a group of isomorphic manipulator neural networks under joint space joint trajectories by building a communication topology, and design the adaptive neural network collaborative learning controller as follows:
Figure GDA0003528357290000171
Figure GDA0003528357290000172
wherein, ci,2The gain matrix is controlled for the designed positive constants,
Figure GDA0003528357290000173
is an estimate of the ideal neural network weight, S (psi)i) Is a vector psiiIs a Gaussian radial basis function of the input, aijIs a contiguous matrix element of the inter-arm communication topology, zi,pBeing error variable, ΓiTo design a positive definite diagonal matrix, σiIs a constant; rho is a cooperative gain coefficient;
time period [ t ] for consistent convergence of closed-loop system after cooperative learning control1,t2]And selecting a constant neural network weight form as follows:
Figure GDA0003528357290000174
designing a constant neural network controller by using the empirical weight:
Figure GDA0003528357290000175
in this embodiment, the training module is configured to utilize the feature extraction and generalization capability of the convolutional neural network in the identification stage, perform feature extraction on a training image through convolutional pooling to obtain a feature layer, perform convolutional pooling on the feature layer to generalize the feature to obtain an enhancement layer, and link the feature layer and the enhancement layer to the output layer in a full link manner to construct a class of convolutional neural network-based width learning system as an image classifier;
in this embodiment, the mapping table setting module is configured to set a mapping table for the image category and the plurality of work tasks in the image classifier in the identification stage, and establish a one-to-one correspondence relationship, which is expressed as:
f:κ→Σy
where κ ∈ {1, 2.., K } is the label of the image classification, ΣyAll work tasks which can be completed by the mechanical arm group;
in this embodiment, the image information acquisition module monitors the environmental change in real time through a camera in the identification stage to obtain an image tag;
in this embodiment, the work task switching module is configured to switch to a work task corresponding to the image tag through an established mapping table in the identification stage;
in this embodiment, the control signal output module is configured to output a control signal, and the mechanical arm calls the constant neural network controller in the recognition stage to complete the multi-task autonomous control of the mechanical arm.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.

Claims (10)

1. A multi-task autonomous learning control method of a mechanical arm based on image recognition is characterized by comprising the following steps:
constructing a kinematic model, a dynamic model and a plurality of expected regression Cartesian space trajectory models of a group of isomorphic mechanical arms in an identification stage, and defining each expected regression trajectory as a work task;
in the identification stage, mapping Cartesian space trajectories into joint space regression trajectories of the mechanical arm according to the constructed inverse kinematics model of the mechanical arm, realizing online experience sharing of weights of a group of isomorphic mechanical arm neural networks under joint space joint trajectories by constructing a communication topological structure, and designing an adaptive neural network collaborative learning controller as follows:
Figure FDA0003574805680000011
Figure FDA0003574805680000012
wherein, ci,2The gain matrix is controlled for the designed positive constants,
Figure FDA0003574805680000013
is an estimate of the ideal neural network weight, S (psi)i) Is a vector psiiAs an input Gaussian radial basis function, aijIs a contiguous matrix element of the inter-arm communication topology, zi,pBeing error variable, ΓiTo design a positive definite diagonal matrix, σiIs a constant; rho is a cooperative gain coefficient;
time period [ t ] for consistent convergence of closed-loop system after cooperative learning control1,t2]And selecting a constant neural network weight form as follows:
Figure FDA0003574805680000014
designing a constant neural network controller by using the empirical weight:
Figure FDA0003574805680000015
in the identification stage, by utilizing the feature extraction and generalization capability of a convolutional neural network, performing feature extraction on a training image through convolutional pooling operation to obtain a feature layer, generalizing the features through the feature layer through convolutional pooling operation to obtain an enhancement layer, and fully linking the feature layer and the enhancement layer to an output layer to construct a width learning system based on the convolutional neural network as an image classifier;
in the identification stage, the image classification and a plurality of work tasks in the image classifier are set with a mapping table, and a one-to-one correspondence relationship is established and expressed as:
f:κ→Σy
where κ ∈ {1, 2.,. K } is the label of the image classification, ΣyAll work tasks which can be completed by the mechanical arm group;
in the identification stage, a camera monitors environmental changes in real time to obtain an image label;
switching to a working task corresponding to the image label through the established mapping table in the identification stage;
and the mechanical arm calls a constant neural network controller in the identification stage to complete the multi-task autonomous control of the mechanical arm.
2. The image recognition-based multi-task autonomous learning control method for the mechanical arm is characterized in that a set of kinematic models of the isomorphic mechanical arm is constructed in the identification stage, and the method comprises the following specific steps:
positive kinematics represents the mapping of the mechanical arm from joint space to cartesian space:
χ＝T(q)
inverse kinematics represents the mapping of the mechanical arm from cartesian space to joint space:
q＝InvT(χ)
wherein, χ ═ x, y, z]TFor the arm tip position in Cartesian space, q ═ θ1,θ2,...,θn]TFor the angular displacement of the mechanical arm in the joint space, n corresponds to the number of joints of the mechanical arm, T (-) is a positive kinematics mapping relation, and InvT (-) is an inverse kinematics mapping relation.
3. The image recognition-based multitask autonomous learning control method for mechanical arms according to claim 1, wherein the dynamical model of the group of isomorphic mechanical arms is represented as:
Figure FDA0003574805680000021
wherein x isi,1＝[xi,1,1,xi,1,2,…,xi,1,n]T、xi,2＝[xi,2,1,xi,2,2,…,xi,2,n]TRespectively, the angular displacement and angular velocity of the joint of the mechanical arm, M (x)i,1) Is the inertia matrix of the arm, C (x)i,1,xi,2) As a centripetal force matrix, G (x)i,1) Is a gravity term, F (x)i,2) Is the friction force vector uiTo control the torque.
4. The image recognition-based multi-task autonomous learning control method for the mechanical arm according to claim 1, wherein the cartesian space trajectory model is expressed as:
Figure FDA0003574805680000031
wherein the content of the first and second substances,
Figure FDA0003574805680000032
the end of the mechanical arm is on the Cartesian under the work task of image classification label kappa respectivelyThe expected regression trajectory in the molar space is,
Figure FDA0003574805680000033
respectively, known as continuously-derivable periodic functions.
5. The image recognition-based multi-task autonomous learning control method for the mechanical arm according to claim 1, wherein the cartesian space trajectory is mapped to a joint space regression trajectory of the mechanical arm according to the constructed inverse kinematics model of the mechanical arm, which is specifically represented as:
qd＝InvT(χd)
wherein, χdRepresenting a Cartesian space trajectory, qdThe regression trajectory of the mechanical arm joint space is shown, and InvT (cndot) shows the inverse kinematics mapping relation.
6. The image recognition-based mechanical arm multitask autonomous learning control method according to claim 1, wherein the training image is subjected to feature extraction through convolution pooling operation to obtain a feature layer, and the specific steps include:
for j input image in training data setj＝Rw0×h0The following convolution and pooling operations were performed:
Figure FDA0003574805680000034
Ti p＝Pooling(Ti c,pf,spf)∈Rw2×h2
wherein, Ti cAnd Ti pRepresenting the output images of the convolution Conv (-) and Pooling Pooling (-) operations, respectively,
Figure FDA0003574805680000035
randomly generated size theta for ith channelf×θf1,2, etaf，ηfIs the total number of channels of the convolution kernel,
Figure FDA0003574805680000036
for the bias term, s, corresponding to the ith channelcf、spfStride, p, selected for convolution and pooling operations, respectivelyfFor pooling windows, w1 × h1, w2 × h2 are the sizes of the output images for performing convolution and pooling operations, respectively;
will { T }i p|i＝1,2,...,ηfRemodeling is the number of characteristic nodes as
Figure FDA00035748056800000414
Feature vector of
Figure FDA0003574805680000041
Figure FDA00035748056800000415
The training samples are processed by convolution pooling operation to obtain a characteristic layer
Figure FDA0003574805680000042
ξ (-) is the tanh or sigmoid activation function, and L is the number of training samples.
7. The image recognition-based mechanical arm multitask autonomous learning control method according to claim 1, wherein the feature layer is subjected to convolution pooling operation, the features are generalized to obtain an enhancement layer, and the specific steps include:
the output image of the convolution and pooling operation is represented as:
Figure FDA0003574805680000043
Figure FDA0003574805680000044
wherein the content of the first and second substances,
Figure FDA0003574805680000045
and
Figure FDA0003574805680000046
output images representing convolution and Pooling operations, respectively, Conv (-) representing a convolution operation, Pooling (-) representing a Pooling operation,
Figure FDA0003574805680000047
randomly generated size theta for ith channele×θe1,2, etae，ηeIs the total number of channels of the convolution kernel,
Figure FDA0003574805680000048
for the bias term, s, corresponding to the ith channelce、speStride, p, selected for convolution and pooling operations, respectivelyeFor pooling windows, w3 × h3, w4 × h4 are the sizes of the output images for performing convolution and pooling operations, respectively;
will be provided with
Figure FDA0003574805680000049
Remodeling as a characteristic number of nodes
Figure FDA00035748056800000410
Feature vector of
Figure FDA00035748056800000411
Figure FDA00035748056800000412
Finally, the enhancement layer is obtained
Figure FDA00035748056800000413
Wherein, L is the number of training samples, and ξ (-) is a tanh or sigmoid activation function.
8. The image recognition-based mechanical arm multitask autonomous learning control method is characterized in that the characteristic layer and the enhancement layer are all linked to the output layer, a convolutional neural network-based width learning system is constructed to serve as an image classifier, and the method comprises the following specific steps:
combining the characteristic nodes and the enhanced nodes into A ═ Z | E ], and in a training stage, calculating a pseudo-inverse value of A by using a ridge regression algorithm:
Figure FDA0003574805680000051
wherein I is the heel matrix ATA is a unit matrix with the same size, lambda is a regular term coefficient in a ridge regression algorithm, Z represents a characteristic layer, and E represents an enhancement layer;
the weight of the output layer can be obtained from Y ═ AW:
W＝A+Y
wherein Y ∈ RL*KFor the output matrix of the training set, L is the number of samples and K is the number of image labels.
9. The image recognition-based multi-task autonomous learning control method for the mechanical arm is characterized in that the camera monitors environmental changes in real time to obtain an image label, and the method comprises the following specific steps:
the camera captures an indication image;
preprocessing a captured image, wherein the preprocessing comprises the steps of region selection, binarization and dimension adjustment;
and taking the preprocessed image as the input of the trained image classifier to obtain the label of the image.
10. A mechanical arm multitask autonomous learning control system based on image recognition is characterized by comprising: the system comprises a model building module, a space trajectory mapping module, a controller building module, a training module, a mapping table setting module, an image information acquisition module, a work task switching module and a control signal output module;
the model construction module is used for constructing a kinematic model, a dynamic model and a plurality of expected regression Cartesian space trajectory models of a group of isomorphic mechanical arms in an identification stage, and each expected regression trajectory is defined as a work task;
the space track mapping module is used for mapping the Cartesian space track into a mechanical arm joint space regression track according to the constructed mechanical arm inverse kinematics model in the identification stage;
the controller construction module is used for realizing online experience sharing of weights of a group of isomorphic mechanical arm neural networks under joint space joint tracks by constructing a communication topological structure, and the self-adaptive neural network collaborative learning controller is designed as follows:
Figure FDA0003574805680000061
Figure FDA0003574805680000062
wherein, ci,2The gain matrix is controlled for the designed positive constants,
Figure FDA0003574805680000063
is an estimate of the ideal neural network weight, S (psi)i) Is a vector psiiIs a Gaussian radial basis function of the input, aijIs a contiguous matrix element of the inter-arm communication topology, zi,pBeing error variable, ΓiTo design a positive definite diagonal matrix, σiIs a constant; rho is a cooperative gain coefficient;
time period [ t ] for consistent convergence of closed-loop system after cooperative learning control1,t2]And selecting a constant neural network weight form as follows:
Figure FDA0003574805680000064
designing a constant neural network controller by using the empirical weight:
Figure FDA0003574805680000065
the training module is used for utilizing the feature extraction and generalization capability of the convolutional neural network in the identification stage, carrying out feature extraction on a training image through convolutional pooling operation to obtain a feature layer, carrying out convolutional pooling operation on the feature layer to generalize the features to obtain an enhancement layer, and fully linking the feature layer and the enhancement layer to an output layer to construct a width learning system based on the convolutional neural network as an image classifier;
the mapping table setting module is used for setting a mapping table for the image category and the plurality of work tasks in the image classifier in the identification stage, and establishing a one-to-one correspondence relationship, which is expressed as:
f:κ→Σy
where κ ∈ {1, 2.,. K } is the label of the image classification, ΣyAll work tasks which can be completed by the mechanical arm group;
the image information acquisition module monitors environmental changes in real time through a camera in an identification stage to obtain an image tag;
the work task switching module is used for switching to the work task corresponding to the image label through the established mapping table in the identification stage;
the control signal output module is used for outputting control signals, and the mechanical arm calls the constant neural network controller in the identification stage to complete the multi-task autonomous control of the mechanical arm.
CN202011373252.0A 2020-11-30 2020-11-30 Image recognition-based mechanical arm multitask autonomous learning control method and system Active CN112381173B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011373252.0A CN112381173B (en) 2020-11-30 2020-11-30 Image recognition-based mechanical arm multitask autonomous learning control method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011373252.0A CN112381173B (en) 2020-11-30 2020-11-30 Image recognition-based mechanical arm multitask autonomous learning control method and system

Publications (2)

Publication Number Publication Date
CN112381173A CN112381173A (en) 2021-02-19
CN112381173B true CN112381173B (en) 2022-06-14

Family

ID=74588671

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011373252.0A Active CN112381173B (en) 2020-11-30 2020-11-30 Image recognition-based mechanical arm multitask autonomous learning control method and system

Country Status (1)

Country Link
CN (1) CN112381173B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102289204A (en) * 2011-06-03 2011-12-21 华南理工大学 Mechanical arm general control method based on determined learning theory
CN109807887A (en) * 2019-01-18 2019-05-28 上海交通大学 Flexible arm Intellisense and control method and system based on deep neural network

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8719212B2 (en) * 2011-05-09 2014-05-06 King Fahd University Of Petroleum And Minerals Parallel kinematic machine trajectory planning method
CN108656107B (en) * 2018-04-04 2020-06-26 北京航空航天大学 Mechanical arm grabbing system and method based on image processing
CN110840666B (en) * 2019-11-19 2022-09-09 华南理工大学 Wheelchair mechanical arm integrated system based on electro-oculogram and machine vision and control method thereof

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102289204A (en) * 2011-06-03 2011-12-21 华南理工大学 Mechanical arm general control method based on determined learning theory
CN109807887A (en) * 2019-01-18 2019-05-28 上海交通大学 Flexible arm Intellisense and control method and system based on deep neural network

Also Published As

Publication number Publication date
CN112381173A (en) 2021-02-19

Similar Documents

Publication Publication Date Title
CN108196453B (en) Intelligent calculation method for mechanical arm motion planning group
CN102915039B (en) A kind of multirobot joint objective method for searching of imitative animal spatial cognition
CN109240091B (en) Underwater robot control method based on reinforcement learning and tracking control method thereof
CN110275436A (en) A kind of RBF neural self-adaptation control method of more single arm robots
CN110000795A (en) A kind of method of Visual servoing control, system and equipment
CN108333949A (en) For the sliding formwork fault tolerant control method of multi-agent system actuator partial failure failure
Kumar et al. Visual motor control of a 7DOF redundant manipulator using redundancy preserving learning network
CN108445768B (en) Augmented self-adaptive fuzzy control method for operation space trajectory tracking of space robot
CN114117926B (en) Robot cooperative control algorithm based on federal learning
CN110070565B (en) Ship track prediction method based on image superposition
CN112207835B (en) Method for realizing double-arm cooperative work task based on teaching learning
Chen et al. Robot learning from multiple demonstrations with dynamic movement primitive
Srivastava et al. Least square policy iteration for ibvs based dynamic target tracking
CN111814823A (en) Transfer learning method based on scene template generation
Ren et al. Fast-learning grasping and pre-grasping via clutter quantization and Q-map masking
CN112381173B (en) Image recognition-based mechanical arm multitask autonomous learning control method and system
Liu et al. Sim-and-real reinforcement learning for manipulation: A consensus-based approach
Li et al. Learning complex assembly skills from kinect based human robot interaction
Gäbert et al. Generation of human-like arm motions using sampling-based motion planning
Yousuf Robust output-feedback formation control design for nonholonomic mobile robot (nmrs)
CN111950690A (en) Efficient reinforcement learning strategy model with self-adaptive capacity
Su et al. Nonlinear visual mapping model for 3-D visual tracking with uncalibrated eye-in-hand robotic system
CN114012733A (en) Mechanical arm control method for scribing PC (personal computer) component mold
Cortes et al. Imitation learning based soft robotic grasping control without precise estimation of target posture
CN113359626A (en) Finite time hierarchical control method for multi-robot system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant