CN116394264B - Group coding impulse neural network-based multi-mechanical arm cooperative motion planning method - Google Patents
Group coding impulse neural network-based multi-mechanical arm cooperative motion planning method Download PDFInfo
- Publication number
- CN116394264B CN116394264B CN202310664822.9A CN202310664822A CN116394264B CN 116394264 B CN116394264 B CN 116394264B CN 202310664822 A CN202310664822 A CN 202310664822A CN 116394264 B CN116394264 B CN 116394264B
- Authority
- CN
- China
- Prior art keywords
- mechanical arm
- neural network
- group
- mechanical
- arm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000033001 locomotion Effects 0.000 title claims abstract description 72
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 63
- 238000000034 method Methods 0.000 title claims abstract description 53
- 210000002569 neuron Anatomy 0.000 claims description 44
- 230000002787 reinforcement Effects 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 12
- 238000013461 design Methods 0.000 claims description 11
- 230000000694 effects Effects 0.000 claims description 9
- 230000001537 neural effect Effects 0.000 claims description 9
- 108091026890 Coding region Proteins 0.000 claims description 8
- 239000000758 substrate Substances 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 230000000638 stimulation Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims 1
- 230000002093 peripheral effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 8
- 238000003860 storage Methods 0.000 description 6
- 238000005265 energy consumption Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 230000000875 corresponding effect Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000007667 floating Methods 0.000 description 2
- 239000012528 membrane Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000000946 synaptic effect Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000011031 large-scale manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1682—Dual arm manipulator; Coordination of several manipulators
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a multi-mechanical arm cooperative motion planning method based on a group coding impulse neural network, which comprises the following steps: any mechanical arm at any moment in motion sceneAnd acquiring state data of the peripheral related mechanical arms, respectively designing a group coding module, a pulse neural network module and a group decoding module, outputting the angle at which each joint of the mechanical arm should rotate at the next moment, and realizing the autonomous motion planning of the mechanical arm. According to the application, through collaborative motion planning of the multi-mechanical arm based on the group coding pulse neural network, each mechanical arm considers the motions of other mechanical arms around the mechanical arm, so that the collision between the mechanical arms can be reduced to a great extent.
Description
Technical Field
The application relates to the technical field of mechanical arm motion planning, in particular to a multi-mechanical arm cooperative motion planning method based on a group coding pulse neural network.
Background
The cooperation of multiple mechanical arms means that multiple mechanical arms cooperate with each other in the same task to achieve higher efficiency and higher precision. In large scale manufacturing processes, such as automotive manufacturing, aerospace manufacturing, etc., multiple robotic arms are often required to work together to complete the various manufacturing steps. In the field of logistics and storage, the cooperative movement of multiple mechanical arms has become one of important means for improving production efficiency, reducing cost and improving quality. In addition, under some dangerous scenes, such as nuclear power stations, dangerous chemical plants and the like, the cooperation of the plurality of mechanical arms can be used for replacing manpower to finish some dangerous operations, so that the production efficiency is improved, and meanwhile, the safety of staff is also ensured.
However, the coordination and consistency of the motion trail of each robot are ensured for the cooperative work of the multiple mechanical arms, so that the task can be efficiently completed. The planning process of the motion trail is a key factor of the cooperative work of the mechanical arms. The motion planning algorithm based on sampling increases with the degree of freedom of the mechanical arm, the calculated amount of the motion planning algorithm increases exponentially, is not rapid and flexible, and lacks the capability of real-time response and dynamic planning. The learning-based motion planning algorithm needs to make trade-offs between a longer training phase and faster online execution, and is mostly focused on solving the motion problem of a single mechanical arm itself, which is insufficient to deal with the problem of co-cooperation between multiple mechanical arms. Existing learning-based multi-arm motion planning studies rely mostly on a centralized Inverse Kinematics (IK) solver, whose constraints must be learned from scratch if arms are added, removed or moved in a work scenario.
The pulsed neural network can be deployed on a neuromorphic processor, which is an energy efficient alternative to deep networks. In the emerging neuromorphic calculation model, memory and calculation are tightly integrated, and only when pulses occur, synaptic weights are added to neuron membrane potentials, so that a large number of floating point number high-precision multiplication operations in a traditional neural network are avoided, and therefore, energy consumption is extremely low in the system operation process. At present, the impulse neural network has been applied to the field of automatic driving environment sensing for detecting, classifying or tracking objects such as automobiles, pedestrians, road signs and the like, but the research of the impulse neural network on the problem of high-dimensional continuous motion planning of an industrial mechanical arm is quite limited, and is getting more attention and investment.
Disclosure of Invention
The application aims to provide a multi-mechanical arm cooperative motion planning method based on a group coding pulse neural network, which aims to solve the problems of low flexibility and high energy consumption in the conventional multi-mechanical arm motion planning.
In order to achieve the above purpose, the present application provides the following technical solutions: a multi-mechanical arm cooperative motion planning method based on a group coding impulse neural network comprises the following steps:
step one, any mechanical arm at any moment in a motion sceneAcquiring state data of related mechanical arms around the robot;
step two, designing a group coding module, and coding each dimension of the observed state data into a neuron group to obtain a pulse signal;
step three, designing a pulse neural network module to process pulse signals and output group activity pulses;
step four, designing a group decoding module, namely decoding group movement pulses into angles at which each joint of the mechanical arm should rotate at the next moment, and realizing autonomous movement planning of the mechanical arm;
and fifthly, optimizing parameters of a group coding module, a pulse neural network module and a group decoding module based on the deep reinforcement learning framework, and planning cooperative movement of a plurality of mechanical arms in a scene.
Preferably, the relevant mechanical arm is all the substrates and mechanical armsThe mechanical arm with the substrate distance within 1.0m comprises a mechanical arm per se, and is a mechanical arm +.>Is provided;
the status data of any one of the relevant mechanical arms includes: base pose, tail end pose, 10 joint positions, 6 joint angles and target tail end pose of the related mechanical arm;
the pose is a position and a pose of the European space, the position is represented by 3 dimension data, the pose is represented by a quaternion, namely 4 dimensions, and the state data of any one related mechanical arm is represented by (7+7+10×3+6+7=57) dimensions.
Preferably, step S2 further includes: for any mechanical armThe coding sequence and coding criteria of the mechanical arm related to the design.
Preferably, the coding sequence of the mechanical arm related to the design comprises the following steps:
substrate-to-robot based on related robotThe distance of the substrate is determined by the far and near sequencing related mechanical arms>Set a of robotic arms that are centered:
；
wherein P is the number of the mechanical arms, the value from 1 to P, P is the number of the related mechanical arms,the last mechanical arm in the set A is shown as the mechanical arm +.>Itself.
Preferably, the coding criteria for designing the mechanical arm related to the coding criteria include:
for any one relative mechanical armAt time->Status of->Is +.>Data of->，/>；
Will firstTransformation into a population +.>Stimulation intensity of individual neurons:
；
wherein ,are trainable parameters; then based on the stimulus intensity->Generating pulse signalsThe generation rule is as follows:
；
；
；
wherein ,，/>is a weight coefficient, takes the value of 0.99, < >>Is an indication equation, each element of the input vector is processed to be greater than a threshold +.>When in use, thenThe corresponding element takes a value of 1, otherwise takes a value of 0, therefore, go through +.>And processing to obtain pulse signals.
Preferably, the impulse neural network module designs impulse neurons based on the circuit leakage-integration-discharge working principle.
Preferably, the impulse neuron is designed based on a circuit leakage-integration-discharge operation principle, and comprises:
firstly, determining an updating rule of any pulse neuron mode voltage as follows:
；
wherein ,representing the number of layers of the neural network in which the neurons are located; />Indicating the layer->A neuron; the neuron initial mode voltage is 0, namely: />；
First, theLayer->The inputs to the individual impulse neurons are:
；
wherein n represents the thNumber of layer pulse neurons, firstThe input of the 1-layer neuron is the pulse signal output by the group coding module, (-) and>) Parameters representing neurons;
for related mechanical armsStatus data of->Layer->The output of the individual impulse neurons is:
。
preferably, the impulse neural network module consists of 3 impulse neural layers which are connected in sequence, and the number of impulse neurons of the impulse neural layers is 256, 128 and 10 respectively.
Preferably, the group motion pulse is weighted, and the pulse signal is decoded into a mechanical armThe real value of the angle by which the 6 joints of (a) should be turned at the next moment, i.e. +.>Angle value of +.>, wherein />Is a parameter of the group decoding module,/>Is a group activity pulse.
Preferably, in step S5, parameters of a group encoding module, a pulse neural network module and a group decoding module are optimized based on a deep reinforcement learning framework, so as to implement collaborative motion planning of a plurality of mechanical arms in a scene, including:
the strategy network in the deep reinforcement learning framework consists of a group coding module, a pulse neural network module and a group decoding module, and the value network consists of the group coding module, the pulse neural network module and a full-connection layer, wherein the output dimension of the full-connection layer is 1;
the reward functions in the training process of the deep reinforcement learning framework are:；
wherein ,rewards representing the arrival of a single arm at the target end position, when the single arm arrives at the target end position,/->Otherwise->；/>Representing rewards for all the mechanical arms reaching the target end pose in the scene, and when all the mechanical arms reach the target end pose, the reward is +.>Otherwise->；/>Indicating a penalty for collision during the planning of the movement of the mechanical arm, when a collision occurs, the +.>Otherwise, let(s)>；
Randomly initializing parameters of the whole reinforcement learning framework, updating the parameters by using an adaptive moment estimation optimization algorithm based on a loss function of the deep reinforcement learning framework until a reward function curve converges, and completing an online training process to obtain three optimized modules;
and deploying the optimized three modules to each mechanical arm in the scene to realize the cooperative motion planning of a plurality of mechanical arms in the scene.
Compared with the prior art, the application has the beneficial effects that:
the multi-mechanical arm cooperative motion planning method based on the group coding pulse neural network can realize the cooperative motion planning of any number of mechanical arms in any working scene, and compared with the traditional mechanical arm motion planning method based on learning, the method has the advantages that each mechanical arm considers the motions of other mechanical arms around the mechanical arm, so that the collision among the mechanical arms can be reduced to a great extent, and the cooperative motion planning of the multi-mechanical arms can be realized;
the application is based on a group coding pulse neural network, and only when pulses appear, the synaptic weight is added to the neuron membrane potential of the pulse neural network, so that a large number of floating point number high-precision multiplication operations in the traditional neural network are avoided, and the energy consumption is extremely low and the pulse neural network is easy to deploy into an embedded system in a robot arm in the running process of the system.
Drawings
FIG. 1 is a main flow chart of a multi-mechanical arm cooperative motion planning method based on a group coding impulse neural network provided by an embodiment of the application;
FIG. 2 is a diagram of a determination and mechanical arm in a multi-mechanical-arm collaborative motion planning method based on a group coding impulse neural network according to an embodiment of the present applicationSchematic diagram of a related mechanical arm;
fig. 3 is a schematic diagram of a pulse neural network module in a multi-mechanical arm cooperative motion planning method based on a group coding pulse neural network according to an embodiment of the present application;
fig. 4 is a schematic diagram of motion planning simulation of multiple mechanical arms in a multi-mechanical-arm collaborative motion planning method based on a group coding impulse neural network according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The main execution body of the method in this embodiment is a terminal, and the terminal may be a device such as a mobile phone, a tablet computer, a PDA, a notebook or a desktop, but of course, may be another device with a similar function, and this embodiment is not limited thereto.
Referring to fig. 1 to 4, the present application provides a multi-mechanical arm cooperative motion planning method based on a group coding impulse neural network, which is applied to multi-mechanical arm cooperative motion planning, and includes:
s1, any mechanical arm at any moment in a motion sceneAcquiring state data of related mechanical arms around the robot;
the correlation between the mechanical arms in S1 is determined by the distance between the substrates of the mechanical arms, and the correlated mechanical arms are all substrates and mechanical armsThe mechanical arm with the substrate distance within 1.0m comprises a mechanical arm per se, and is a mechanical arm +.>Is provided; referring to fig. 2, fig. 2 is a schematic diagram of a group coding impulse neural network-based multi-mechanical arm collaborative motion planning method according to an embodiment of the present applicationDetermining and arm->Schematic diagram of a related mechanical arm;
wherein, the state data of any one relevant mechanical arm comprises: base pose, tail end pose, 10 joint positions, 6 joint angles and target tail end pose of the related mechanical arm; the pose is the position and the pose of the European space, the position is represented by 3 dimension data, the pose is represented by quaternion, namely 4 dimensions, and the state data of any one related mechanical arm is represented by (7+7+10×3+6+7=57) dimensions.
S2, designing a group coding module, and coding each dimension of the observed state data into a neuron group to obtain a pulse signal;
wherein S2 further comprises: for any mechanical armThe coding sequence and coding criteria of the mechanical arm related to the design.
Specifically, the method comprises the following steps:
s21, designing the coding sequence of the mechanical arm related to the coding sequence, wherein the coding sequence comprises the following steps:
substrate-to-robot based on related robotThe distance of the substrate is determined by the far and near sequencing related mechanical arms>Set a of robotic arms that are centered:
；
wherein P is the number of the mechanical arms, the value from 1 to P, P is the number of the related mechanical arms,the last mechanical arm in the set A is shown as the mechanical arm +.>Itself.
S22, designing coding criteria of the mechanical arm related to the design criteria, wherein the coding criteria comprise:
for any one relative mechanical armAt time->Status of->Is +.>Data of->，/>；
Will firstTransformation into a population +.>Stimulation intensity of individual neurons:
；
wherein ,are trainable parameters; then based on the stimulus intensity->Generating pulse signalsThe generation rule is as follows:
；
；
；
wherein ,，/>is a weight coefficient, takes the value of 0.99, < >>Is an indication equation, each element of the input vector is processed to be greater than a threshold +.>If so, the corresponding element takes a value of 1, otherwise takes a value of 0, thus, go through +.>And processing to obtain pulse signals.
S3, designing a pulse neural network module to process the pulse signals and output group activity pulses;
the impulse neural network module designs impulse neurons based on a circuit leakage-integration-discharge working principle.
Specifically, the method comprises the following steps:
s31, designing a pulse neuron based on a circuit leakage-integration-discharge working principle, comprising:
firstly, determining an updating rule of any pulse neuron mode voltage as follows:
；
wherein ,representing the number of layers of the neural network in which the neurons are located; />Indicating the layer->A neuron; the neuron initial mode voltage is 0, namely: />；
First, theLayer->The inputs to the individual impulse neurons are:
；
wherein n represents the thThe number of the layer pulse neurons, namely the input of the 1 st layer neuron is the pulse signal output by the group coding module, (-) and%>) Parameters representing neurons;
for related mechanical armsStatus data of->Layer->The output of the individual impulse neurons is:
。
s32, constructing a pulse neural network module comprises the following steps:
the impulse neural network module consists of 3 impulse neural layers which are sequentially connected, and comprises a first impulse neural layer, a second impulse neural layer and a third impulse neural layer, wherein the number of impulse neurons of each impulse neural layer is 256, 128 and 10 respectively in sequence, please refer to fig. 3, and fig. 3 is a schematic diagram of the impulse neural network module in the multi-mechanical arm cooperative motion planning method based on the group coding impulse neural network provided by the embodiment of the application.
S4, designing a group decoding module to decode group activity pulses into mechanical armsThe angle that each joint should rotate at the next moment realizes the autonomous motion planning of the mechanical arm;
specifically, S4 includes: weighting group activity pulse, decoding pulse signal into mechanical armThe real value of the angle by which the 6 joints of (a) should be turned at the next moment, i.e. +.>Angle value of (2), wherein />Is a parameter of the group decoding module,/>Is a group activity pulse.
And S5, optimizing parameters of a group coding module, a pulse neural network module and a group decoding module based on a deep reinforcement learning framework (SAC), and planning cooperative movement of a plurality of mechanical arms in a scene.
Specifically, S5 includes:
s51, a strategy network in a deep reinforcement learning framework (SAC) consists of a group coding module, a pulse neural network module and a group decoding module, wherein the value network consists of the group coding module, the pulse neural network module and a full-connection layer, and the output dimension of the full-connection layer is 1;
s52, rewarding functions in the training process of the deep reinforcement learning frame (SAC) are as follows:
；
wherein ,rewards representing the arrival of a single arm at the target end position, when the single arm arrives at the target end position,/->Otherwise->；/>Representing rewards for all the mechanical arms reaching the target end pose in the scene, and when all the mechanical arms reach the target end pose, the reward is +.>Otherwise->；/>Indicating a penalty for collision during the planning of the movement of the mechanical arm, when a collision occurs, the +.>Otherwise, let(s)>；
S53, randomly initializing parameters of the whole reinforcement learning frame (SAC), updating the parameters by using an adaptive moment estimation optimization algorithm (Adam) based on a loss function of the learning frame (SAC) until a reward function curve converges, and completing an online training process to obtain three optimized modules;
and S54, deploying the optimized three modules on each mechanical arm in the scene to realize the cooperative motion planning of a plurality of mechanical arms in the scene, wherein as shown in fig. 4, fig. 4 is a simulation schematic diagram of the motion planning of the three mechanical arms in the multi-mechanical-arm cooperative motion planning method based on the group coding pulse neural network, and the numbers of the mechanical arms are represented in the figure.
In this embodiment, each mechanical arm considers the movements of other mechanical arms around the mechanical arm, so that the collision between the mechanical arms can be reduced to a great extent, and thus the collaborative motion planning of multiple mechanical arms can be realized.
In a multi-mechanical-arm working scene, the target end pose of each mechanical arm is known, and by adopting the multi-mechanical-arm cooperative motion planning method based on the group coding pulse neural network, each mechanical arm in the scene can avoid mechanical arms of other motions, smoothly reaches the target pose, and the energy consumption in the whole motion planning process is lower. The multi-mechanical-arm cooperative motion planning method based on the group coding pulse neural network enables any number of mechanical arms in a working scene to reach a target pose under the condition of no collision, realizes co-cooperation among the multi-mechanical arms and ensures lower system energy consumption.
On the basis of the embodiment, the application further provides electronic equipment, which comprises:
the device comprises a processor and a memory, wherein the processor is in communication connection with the memory;
in this embodiment, the memory may be implemented in any suitable manner, for example: the memory can be read-only memory, mechanical hard disk, solid state hard disk, USB flash disk or the like; the memory is used for storing executable instructions executed by at least one processor;
in this embodiment, the processor may be implemented in any suitable manner, e.g., the processor may take the form of, for example, a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), a programmable logic controller, and an embedded microcontroller, etc.; the processor is configured to execute executable instructions to implement a multi-manipulator collaborative motion planning method based on a group-encoded impulse neural network as described above.
On the basis of the embodiment, the application also provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program realizes the multi-mechanical arm cooperative motion planning method based on the group coding pulse neural network when being executed by a processor.
Those of ordinary skill in the art will appreciate that the modules and method steps of the examples described in connection with the embodiments disclosed herein can be implemented as electronic hardware, or as a combination of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus, device and module described above may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules is merely a logical function division, and there may be additional divisions of actual implementation, e.g., multiple modules or units may be combined or integrated into another apparatus, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or apparatuses, which may be in electrical, mechanical or other form.
The modules illustrated as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored on a computer readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a read-only memory server, a random access memory server, a magnetic disk or an optical disk, or other various media capable of storing program instructions.
In addition, it should be noted that the combination of the technical features described in the present application is not limited to the combination described in the claims or the combination described in the specific embodiments, and all the technical features described in the present application may be freely combined or combined in any manner unless contradiction occurs between them.
It should be noted that the above-mentioned embodiments are merely examples of the present application, and it is obvious that the present application is not limited to the above-mentioned embodiments, and many similar variations are possible. All modifications attainable or obvious from the present disclosure set forth herein should be deemed to be within the scope of the present disclosure.
The foregoing is merely illustrative of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (8)
1. A multi-mechanical arm cooperative motion planning method based on a group coding impulse neural network is characterized by comprising the following steps:
s1, any mechanical arm at any moment in a motion scene k Acquiring state data of related mechanical arms around the robot;
s2, designing a group coding module, and coding each dimension of the observed state data into a neuron group to obtain a pulse signal;
s3, designing a pulse neural network module to process the pulse signals and output group activity pulses;
the impulse neural network module designs impulse neurons based on a circuit leakage-integration-discharge working principle, and specifically comprises the following components:
firstly, determining an updating rule of any pulse neuron mode voltage as follows:
wherein, P is the number of the mechanical arms, the value from 1 to P, and P is the number of the related mechanical arms; θ (·) is an indication equation, processing each element of the input vector, greater than a threshold V th When the corresponding element takes a value of 1, otherwise, takes a value of 0; v (V) th Is the pulse neuron mode voltage threshold; l represents the number of layers of the neural network in which the neurons are located; m represents the mth neuron of the layer; the neuron initial mode voltage is 0, namely:
the inputs to the mth impulse neuron of layer l are:
wherein n represents the number of the first-1 layer pulse neurons, the input of the first layer neurons is the pulse signal output by the group coding module,parameters representing neurons;
arm for related mechanical arm (p) The output of the mth pulse neuron of the first layer is:
s4, designing a group decoding module, and decoding group activity pulses into angles at which each joint of the mechanical arm should rotate at the next moment, so as to realize autonomous motion planning of the mechanical arm;
and S5, optimizing parameters of a group coding module, a pulse neural network module and a group decoding module based on the deep reinforcement learning framework, and planning cooperative movement of a plurality of mechanical arms in a scene.
2. Group-encoded impulse-based neural network of claim 1The multi-mechanical arm cooperative motion planning method is characterized in that the related mechanical arms are all substrates and mechanical arm k A robotic arm having a base distance in the range of 1.0m and including the robotic arm itself;
the status data of any one of the relevant mechanical arms includes: base pose, tail end pose, 10 joint positions, 6 joint angles and target tail end pose of the related mechanical arm;
the pose is a position and a pose of the European space, the position is represented by 3 dimension data, the pose is represented by a quaternion, namely 4 dimensions, and the state data of any one related mechanical arm is represented by 7+7+10×3+6+7=57 dimensions.
3. The method for planning the coordinated motion of multiple mechanical arms based on the group-encoded impulse neural network according to claim 1, wherein in the step S2, the method further comprises: arm for any mechanical arm k The coding sequence and coding criteria of the mechanical arm related to the design.
4. A multi-robot collaborative motion planning method based on a group coded impulse neural network according to claim 3, wherein the design and coding sequence of its associated robots comprises:
based on the distance from the base of the relevant mechanical arm to the base of the mechanical arm, the relevant mechanical arm is sequenced from far to near to obtain arm k Set a of robotic arms that are centered:
wherein P is the number of the mechanical arms, and can be from 1 to P, P is the number of the relevant mechanical arms, arm (p) Representing the last mechanical arm in the set A as the arm k Itself.
5. A multi-robot collaborative motion planning method based on a group coded impulse neural network according to claim 3, wherein the design and coding criteria of its associated robot comprises:
for any one relative mechanical arm (p) E A at time t i State of (2)Data of any one dimension j
Will firstConversion to stimulation intensity of L neurons in a population:
wherein ,are trainable parameters; then based on the stimulus intensity->Generating a pulse signal +.>The generation rule is as follows:
wherein ,alpha is a weight coefficient, the value is 0.99, theta (·) is an indication equation, each element of the input vector is processed to be greater than a threshold value V th When the pulse signal is processed, the corresponding element takes a value of 1, otherwise takes a value of 0, so that the pulse signal can be obtained through theta (·) processing.
6. The group coding impulse neural network-based multi-mechanical arm collaborative motion planning method according to claim 1, wherein the impulse neural network module is composed of 3 impulse neural layers which are connected in sequence, and the number of impulse neurons of the impulse neural layers is 256, 128 and 10 respectively in sequence.
7. The method for planning the coordinated motion of multiple mechanical arms based on the group-encoded impulse neural network according to claim 1, wherein the group-active impulses are weighted to decode impulse signals into mechanical arm k The real angle value of the 6 joints to be rotated at the next moment, i.e. the angle value a for any joint q q ＝W q ·δ+b q, wherein (Wq ，b q ) Is a parameter of the group decoding module, and δ is the group activity pulse.
8. The method for planning the coordinated motion of multiple mechanical arms based on the group coding impulse neural network according to claim 1, wherein the optimizing parameters of the group coding module, the impulse neural network module and the group decoding module based on the deep reinforcement learning framework in the step S5, to implement the coordinated motion planning of multiple mechanical arms in a scene, includes:
the strategy network in the deep reinforcement learning framework consists of a group coding module, a pulse neural network module and a group decoding module, and the value network consists of the group coding module, the pulse neural network module and a full-connection layer, wherein the output dimension of the full-connection layer is 1;
the reward function in the training process of the deep reinforcement learning framework is as follows:
r t ＝r g +r all +r c ；
wherein ,rg Representing rewards for reaching a target end pose by a single robotic arm, r when the single robotic arm reaches the target end pose g =0.01, otherwise r g ＝0；r all Representing rewards for all mechanical arms reaching the target end pose in a scene, and r is when all mechanical arms reach the target end pose all =1.0, otherwise r all ＝0；r c Indicating punishment of collision in the process of planning movement of mechanical arm, and when collision occurs, r c = -0.05, otherwise, r c ＝0；
Randomly initializing parameters of the whole reinforcement learning framework, updating the parameters by using an adaptive moment estimation optimization algorithm based on a loss function of the deep reinforcement learning framework until a reward function curve converges, and completing an online training process to obtain three optimized modules;
and deploying the optimized three modules to each mechanical arm in the scene to realize the cooperative motion planning of a plurality of mechanical arms in the scene.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310664822.9A CN116394264B (en) | 2023-06-07 | 2023-06-07 | Group coding impulse neural network-based multi-mechanical arm cooperative motion planning method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310664822.9A CN116394264B (en) | 2023-06-07 | 2023-06-07 | Group coding impulse neural network-based multi-mechanical arm cooperative motion planning method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116394264A CN116394264A (en) | 2023-07-07 |
| CN116394264B true CN116394264B (en) | 2023-08-25 |
Family
ID=87009048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310664822.9A Active CN116394264B (en) | 2023-06-07 | 2023-06-07 | Group coding impulse neural network-based multi-mechanical arm cooperative motion planning method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116394264B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116989800B (en) * | 2023-09-27 | 2023-12-15 | 安徽大学 | Mobile robot visual navigation decision-making method based on pulse reinforcement learning |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111612136A (en) * | 2020-05-25 | 2020-09-01 | 之江实验室 | Neural morphology visual target classification method and system |
| CN111923039A (en) * | 2020-07-14 | 2020-11-13 | 西北工业大学 | Redundant mechanical arm path planning method based on reinforcement learning |
| CN112699956A (en) * | 2021-01-08 | 2021-04-23 | 西安交通大学 | Neural morphology visual target classification method based on improved impulse neural network |
| CN114372563A (en) * | 2022-01-10 | 2022-04-19 | 四川大学 | Robot control method and system based on hybrid pulse reinforcement learning network structure |
| CN115438776A (en) * | 2022-10-10 | 2022-12-06 | 西安交通大学 | Coding method for impulse neural network |
| CN115870988A (en) * | 2022-12-29 | 2023-03-31 | 西安交通大学 | Multi-agent cooperative control mechanical arm automatic filling motion planning method |
| CN115908896A (en) * | 2022-10-28 | 2023-04-04 | 北京理工大学 | Image identification system based on impulse neural network with self-attention mechanism |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180137408A1 (en) * | 2015-05-22 | 2018-05-17 | Socopra Sciences Et Genie S.E.C. | Method and system for event-based neural networks |
-
2023
- 2023-06-07 CN CN202310664822.9A patent/CN116394264B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111612136A (en) * | 2020-05-25 | 2020-09-01 | 之江实验室 | Neural morphology visual target classification method and system |
| CN111923039A (en) * | 2020-07-14 | 2020-11-13 | 西北工业大学 | Redundant mechanical arm path planning method based on reinforcement learning |
| CN112699956A (en) * | 2021-01-08 | 2021-04-23 | 西安交通大学 | Neural morphology visual target classification method based on improved impulse neural network |
| CN114372563A (en) * | 2022-01-10 | 2022-04-19 | 四川大学 | Robot control method and system based on hybrid pulse reinforcement learning network structure |
| CN115438776A (en) * | 2022-10-10 | 2022-12-06 | 西安交通大学 | Coding method for impulse neural network |
| CN115908896A (en) * | 2022-10-28 | 2023-04-04 | 北京理工大学 | Image identification system based on impulse neural network with self-attention mechanism |
| CN115870988A (en) * | 2022-12-29 | 2023-03-31 | 西安交通大学 | Multi-agent cooperative control mechanical arm automatic filling motion planning method |
Non-Patent Citations (1)
| Title |
|---|
| 基于脉冲神经网络的目标检测技术研究;韩雯慧;《基于脉冲神经网络的目标检测技术研究》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116394264A (en) | 2023-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Karkus et al. | Qmdp-net: Deep learning for planning under partial observability | |
| US9367798B2 (en) | Spiking neuron network adaptive control apparatus and methods | |
| US9189730B1 (en) | Modulated stochasticity spiking neuron network controller apparatus and methods | |
| WO2019091020A1 (en) | Weight data storage method, and neural network processor based on method | |
| US20200139540A1 (en) | Reduced degree of freedom robotic controller apparatus and methods | |
| US9082079B1 (en) | Proportional-integral-derivative controller effecting expansion kernels comprising a plurality of spiking neurons associated with a plurality of receptive fields | |
| US9008840B1 (en) | Apparatus and methods for reinforcement-guided supervised learning | |
| US20170203437A1 (en) | Apparatus and methods for operating robotic devices using selective state space training | |
| CN116394264B (en) | Group coding impulse neural network-based multi-mechanical arm cooperative motion planning method | |
| US20150127149A1 (en) | Apparatus and methods for online training of robots | |
| WO2013184688A1 (en) | Stochastic apparatus and methods for implementing generalized learning rules | |
| Jalali et al. | Optimal autonomous driving through deep imitation learning and neuroevolution | |
| Chao et al. | Generative adversarial nets in robotic Chinese calligraphy | |
| US11279025B2 (en) | Apparatus and methods for operating robotic devices using selective state space training | |
| Vuković et al. | Trajectory learning and reproduction for differential drive mobile robots based on GMM/HMM and dynamic time warping using learning from demonstration framework | |
| CN114169394A (en) | Multi-variable time series prediction method for multi-scale adaptive graph learning | |
| Fan et al. | Learning resilient behaviors for navigation under uncertainty | |
| CN114077258A (en) | Unmanned ship pose control method based on reinforcement learning PPO2 algorithm | |
| Zhang et al. | Auto-conditioned recurrent mixture density networks for learning generalizable robot skills | |
| CN113743287B (en) | Robot self-adaptive grabbing control method and system based on impulse neural network | |
| Liu et al. | Robotic cognitive behavior control based on biology-inspired episodic memory | |
| Sykora et al. | Artificial Neural Networks in Educational Process | |
| Murata et al. | Learning and recognition of multiple fluctuating temporal patterns using S-CTRNN | |
| Yamada et al. | Attractor representations of language-behavior structure in a recurrent neural network for human-robot interaction | |
| Li et al. | Achieving different levels of adaptability for human robot collaboration utilizing a neuro-dynamical system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |