CN117021111A - Robot part grabbing method based on deep reinforcement learning - Google Patents

Robot part grabbing method based on deep reinforcement learning Download PDF

Info

Publication number
CN117021111A
CN117021111A CN202311164870.8A CN202311164870A CN117021111A CN 117021111 A CN117021111 A CN 117021111A CN 202311164870 A CN202311164870 A CN 202311164870A CN 117021111 A CN117021111 A CN 117021111A
Authority
CN
China
Prior art keywords
robot
workpiece
grabbing
reinforcement learning
deep reinforcement
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311164870.8A
Other languages
Chinese (zh)
Inventor
陈科尹
熊珍连
罗军建
刘伟忠
冯广强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiaying University
Original Assignee
Jiaying University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiaying University filed Critical Jiaying University
Priority to CN202311164870.8A priority Critical patent/CN117021111A/en
Publication of CN117021111A publication Critical patent/CN117021111A/en
Pending legal-status Critical Current

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1628Programme controls characterised by the control loop
    • B25J9/163Programme controls characterised by the control loop learning, adaptive, model based, rule based expert control
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1602Programme controls characterised by the control system, structure, architecture
    • B25J9/161Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1628Programme controls characterised by the control loop
    • B25J9/1653Programme controls characterised by the control loop parameters identification, estimation, stiffness, accuracy, error analysis
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1656Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1661Programme controls characterised by programming, planning systems for manipulators characterised by task planning, object-oriented languages

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Manipulator (AREA)

Abstract

The invention provides a robot part grabbing method based on deep reinforcement learning, and relates to the technical field of robot grabbing. The robot workpiece grabbing method based on deep reinforcement learning comprises the steps of three-dimensional modeling of unordered irregular workpieces, workpiece identification and pose calculation, calibration of the eyes of a robot and grabbing track, and research on an intelligent robot grabbing software and hardware system. According to the invention, three-dimensional information of the target part is obtained by utilizing the structured light camera sensor, the pose of the target part is estimated by utilizing the three-dimensional target recognition positioning technology, the pose of the part is transformed according to the conversion relation determined by hand-eye calibration and is fed back to the mechanical arm, and the mechanical arm executes the final grabbing action according to the feedback information.

Description

Robot part grabbing method based on deep reinforcement learning
Technical Field
The invention relates to the technical field of robot grabbing, in particular to a robot part grabbing method based on deep reinforcement learning.
Background
Robots are largely introduced in the field of industrial production to replace manual work so as to meet the requirements of product updating and diversification. The robot technology is applied to actual production, so that the quality and efficiency of products are improved, the production environment is improved, and the waste of human resources is reduced. In practical automation lines, most robotic techniques for gripping parts still use teaching programming methods. Although the working position of the robot is basically fixed and the working speed is high, the grabbing failure is caused if the position of the target part is changed, and the actual efficiency of the method is low. In recent years, the combined application of machine vision and a robot in industrial production has become a trend, and the spatial position of a target part can be rapidly detected and acquired by utilizing a machine vision technology to guide the robot to complete a grabbing task. At present, the method is widely applied to various fields of product sorting, defect detection, automatic assembly and the like.
In practice, the target parts may be placed in order on a flat surface, but in some other fields, the target parts may be randomly stacked. The traditional method adopts a vibration screening mechanism, an escapement mechanism and the like to arrange target parts in sequence in a mechanical mode, and then adopts algorithm programming to guide a robot to complete the grabbing task. However, the conventional method can only sort specific parts, cannot meet the requirement of mass automatic production in real time, and cannot be widely suitable for flexible automatic production. Therefore, in actual production, the recognition of scattered and stacked parts and the estimation of the pose are urgently needed. The implementation of robotic automation requires many challenges, especially in its own sensing system and complex field environments. The traditional object identification is mainly based on a two-dimensional image, however, the two-dimensional image of the object cannot provide space position information, and the characteristic information of the target part cannot be stably detected due to factors such as illumination transformation, visual angle transformation and the like, so that the uncertainty of a detection result is greatly increased.
The three-dimensional detection system composed of the high-performance vision sensor can obtain the geometric information of the target part, but the high price restricts the popularization and application of the three-dimensional detection system in actual production and life. In recent years, the advent of inexpensive structured light camera sensors and related technologies has brought about dawn for reducing the cost of 3D vision-based detection systems. Meanwhile, a large number of stable three-dimensional visual algorithms are provided, so that the requirement of a visual detection system on the precision of the sensor is reduced, and the defect of low-cost sensor precision is overcome. Under the background, the problem of filtering and denoising of depth data and three-dimensional object detection and pose estimation under a complex environment are studied by taking the structured light camera sensor as an information providing source, so that the possibility of realizing automatic operation of a robot is provided, and the method has important practical significance for promoting further development of intelligent manufacturing.
Disclosure of Invention
(one) solving the technical problems
Aiming at the defects of the prior art, the invention provides a robot part grabbing method based on deep reinforcement learning, which is characterized in that three-dimensional information of a target part is acquired by utilizing a structured light camera sensor, the pose of the target part is estimated by a three-dimensional target recognition positioning technology, the pose of the part is transformed according to a conversion relation determined by hand-eye calibration and is fed back to a mechanical arm, and the mechanical arm executes final grabbing action according to the feedback information.
(II) technical scheme
In order to achieve the above purpose, the invention is realized by the following technical scheme: a robot workpiece grabbing method based on deep reinforcement learning comprises the following steps:
step one: three-dimensional modeling is carried out on unordered irregular workpieces:
a binocular three-dimensional imaging and 3D camera of surface structured light is adopted to acquire three-dimensional coordinates and gray data of a workpiece, and three-dimensional modeling is carried out on the workpiece based on the data;
step two: and (3) identifying and calculating the pose of the workpiece:
based on three-dimensional modeling of the workpiece, the identification and the positioning of the workpiece are researched by utilizing the invariant characteristic of the local characteristics of the workpiece and adopting a convolutional neural network frame, and the pose calculation of the workpiece is researched on the basis of the parallax principle of three-dimensional vision;
step three: calibrating the hand and eye of the robot and grabbing the track:
a small ball calibration method is adopted to research the aspect of robot hand-eye calibration; adopting a deep reinforcement learning theory to study the robot grabbing track planning;
step four: exploration is carried out on an intelligent grabbing software and hardware system of the robot:
a software and hardware system for intelligent grabbing of unordered irregular workpieces in universal wheel industry by a robot is developed.
Preferably, in the fourth step, the software and hardware system includes a hardware layer system and a software layer system.
Preferably, the hardware layer system comprises a robot, a 3D camera, a computer, a pneumatic clamp, an air compressor.
Preferably, the software layer system comprises a calibration module, an image processing module, a point cloud processing module, a communication module and a display module.
Preferably, the calibration module is used for calibrating internal and external parameters of the camera, distortion parameters of the lens and the positional relationship between the camera and the robot coordinate system.
Preferably, the image processing module is used for acquiring an image, filtering and dividing the acquired image, separating a background from a workpiece image, dividing a projection grating projected on the surface of the workpiece, matching the same coding area, controlling the time sequence of the projector and the camera, and reconstructing the point cloud in three dimensions.
Preferably, the point cloud processing module is used for extracting three-dimensional point cloud key points and describing the key points; and establishing a template library, estimating the rotation and translation relation between the template workpiece and the real workpiece by matching key points of the template point cloud with key points of the workpiece point cloud in the real scene, and finally obtaining the robot grabbing pose through coordinate conversion.
Preferably, the communication module is used for taking the real pose of the workpiece calculated by the point cloud processing module as input, calculating the final picking pose of the tail end clamp of the robot through coordinate transformation, sending planned discrete track points to the robot through WiFi communication, driving the robot to reach the corresponding pose, and finally controlling the pneumatic clamp to be closed to grasp the part through Modbus communication.
Preferably, the display module is used for real-time display of the acquired image and display of the point cloud processing module.
(III) beneficial effects
The invention provides a robot part grabbing method based on deep reinforcement learning. The device comprises the following
The beneficial effects are that:
1. and acquiring three-dimensional information of the target part by using a structured light camera sensor, estimating the pose of the target part by using a three-dimensional target recognition positioning technology, transforming the pose of the part according to a transformation relation determined by hand-eye calibration, feeding back to the mechanical arm, and executing a final grabbing action by the mechanical arm according to the feedback information.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention;
FIG. 2 is a schematic diagram of a three-dimensional modeling flow of the present invention;
FIG. 3 is a schematic diagram of workpiece positioning and pose calculation according to the present invention;
FIG. 4 is a schematic diagram of hand-eye calibration and grasping trajectories according to the present invention;
FIG. 5 is a schematic diagram of a software layer system architecture of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Examples:
the embodiment of the invention provides a robot part grabbing method based on deep reinforcement learning, which is shown in fig. 1-4 and comprises the following steps:
step one: three-dimensional modeling is carried out on unordered irregular workpieces:
in order to realize pose calculation and grabbing track planning of the part, the whole three-dimensional modeling is required to be carried out on the part, a 3D camera with binocular three-dimensional imaging and surface structured light is to be adopted, three-dimensional coordinates and gray data of the part are obtained, and three-dimensional modeling is carried out on the part based on the data;
step two: and (3) identifying and calculating the pose of the workpiece:
based on the three-dimensional modeling of the part, the identification and the positioning of the part are researched by utilizing the invariant characteristic of the local characteristics of the part based on a convolutional neural network frame, and the pose calculation of the part is researched based on the parallax principle of three-dimensional vision;
step three: calibrating the hand and eye of the robot and grabbing the track:
in order to achieve part grabbing, three-dimensional coordinates of the part acquired by the 3D camera are unified under a robot coordinate system, so that hand-eye calibration between the 3D camera and the robot is required. The ball calibration method is adopted to study the calibration aspect of the robot hand and eye. In order to realize that the robot avoids obstacles and space singular points, track planning is an indispensable step, and the robot grabbing track planning is researched by adopting a theory based on deep reinforcement learning;
step four: exploration is carried out on an intelligent grabbing software and hardware system of the robot:
a software and hardware system for intelligent grabbing of unordered irregular workpieces in universal wheel industry by a robot is developed.
Specifically, software and hardware system functions are developed based on OpenCV and PCL libraries. Is mainly divided into four parts; the system comprises a hardware layer, a bottom layer software dependency library, a middle functional module and an upper functional module. The hardware layer mainly comprises: 3D cameras, robots, vision systems, projectors, etc. The underlying software dependent libraries are libraries that are used when the system is developed. The intermediate function module includes: the system comprises an image processing module, a point cloud processing module, a calibration module and a robot communication module. The upper layer functional module is used for developing the upper layer function based on the middle functional module, and the main functions of the system are reconstructing point cloud, identifying parts and driving the robot to grasp the parts. Hereinafter, description is mainly made on a hardware layer system and a software layer system.
(1) Hardware layer system design:
the hardware layer system mainly comprises a robot, a 3D camera, a computer, a pneumatic clamp, an air compressor and the like:
1) And (3) a robot: the robot is a small six-axis robot, the dead weight is only 16kg, and the effective load is 4kg. The system comprises a demonstrator and a mechanical body, and can be controlled through WiFi communication.
2) 3D camera: the 3D imaging principle is various, and various products are available from binocular vision simulating human eye mechanisms, to various technical routes based on trigonometry, including two main flows of structured light and line laser, and even to the time of flight (TOF) principle nowadays. Currently commercially available 3D imaging sensors for industry still have line lasers and surface structured light as the main sources. The 3D imaging based on the surface structured light needs to use coding structured light or speckle structured light, the projection method of the structured light is divided into DMD (Digital Micromirror Device), and the MEMS (Micro-Electro Mechanical Systems) galvanometer generates stripe light or uses DOE (Diffraction optical device) to generate speckle structured light. At present, the high-precision industrial grade 3D camera mainly uses DMD to generate coded structured light, and the MEMS galvanometer mode is gradually used for scenes with lower precision requirements due to small volume, low power consumption, low cost and the like, and the DOE mode occupies the main stream in consumer grade products.
3) Pneumatic clamp: the robot adopts a two-claw pneumatic clamp to finish grabbing parts, the aperture is 6mm, the clamping force is 3.7N, the distance is 12mm when the claws are opened, and the distance is 8mm when the claws are closed. The size of the part grabbing position is 10mm, smaller than the opening distance and larger than the closing distance, and gaskets are not needed.
4) Selection and design of other hardware devices: and (3) a computer: the system selects a Huashuo desktop computer, 8G memory, intel smart processor, frequency of 3.6GHZ and hard disk 500G, and configuration basically meets the requirements. Experiment frame: the system designs an experiment frame which is fixed on a workbench of the robot, and the position between the robot and the vision system is limited. Part case: in order to prevent the reflection of the light source inside the parts box from interfering and affecting the image, the inside of the parts box is polished by using frosted paper, so that the bottom plate has a diffuse reflection effect.
(2) Software layer system design:
the development platform of the program is VS2015, the C++ programming language is used, the OpenCV and PCL libraries are used for developing the system functions by adopting an object-oriented modularized design method, as shown in FIG. 5, the program comprises five functional modules, namely a calibration module, an image processing module, a point cloud processing module, a communication module and a display module.
1) And (3) a calibration module: the calibration module is mainly used for calibrating internal and external parameters of the camera and distortion parameters of the lens and calibrating the position relationship between the camera and a robot coordinate system.
2) An image processing module: the image processing module comprises the steps of obtaining an image, filtering and dividing the obtained image, separating a background from a part image, dividing a projection grating projected on the surface of the part, and matching the same coding region. And controlling the time sequence of the projector and the camera, and reconstructing the point cloud in a three-dimensional way.
3) And the point cloud processing module is used for: the point cloud processing module is a core module of the whole system, and mainly completes the function of extracting three-dimensional point cloud key points and describing keys through description operators. And establishing a template library, estimating the rotation and translation relation between the template part and the real part by matching key points of the template point cloud with key points of the part point cloud in the real scene, and finally obtaining the grabbing pose of the robot through coordinate conversion.
4) And a communication module: the robot communication module mainly takes the real pose of the part calculated by the point cloud processing module as input, calculates the final picking pose of the tail end clamp of the robot through coordinate transformation, sends planned discrete track points to the robot through WiFi communication, drives the robot to reach the corresponding pose, and finally controls the pneumatic clamp to be closed and grabbed through Modbus communication.
5) And a display module: the display module mainly comprises real-time display of the chocolate acquired image and display of the point cloud processing module.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (9)

1. A robot part grabbing method based on deep reinforcement learning is characterized by comprising the following steps of: the method comprises the following steps:
step one: three-dimensional modeling is carried out on unordered irregular workpieces:
a binocular three-dimensional imaging and 3D camera of surface structured light is adopted to acquire three-dimensional coordinates and gray data of a workpiece, and three-dimensional modeling is carried out on the workpiece based on the data;
step two: and (3) identifying and calculating the pose of the workpiece:
based on three-dimensional modeling of the workpiece, the identification and the positioning of the workpiece are researched by utilizing the invariant characteristic of the local characteristics of the workpiece and adopting a convolutional neural network frame, and the pose calculation of the workpiece is researched on the basis of the parallax principle of three-dimensional vision;
step three: calibrating the hand and eye of the robot and grabbing the track:
a small ball calibration method is adopted to research the aspect of robot hand-eye calibration; adopting a deep reinforcement learning theory to study the robot grabbing track planning;
step four: exploration is carried out on an intelligent grabbing software and hardware system of the robot:
a software and hardware system for intelligent grabbing of unordered irregular workpieces in universal wheel industry by a robot is developed.
2. The robot gripping method based on deep reinforcement learning of claim 1, wherein: in the fourth step, the software and hardware system comprises a hardware layer system and a software layer system.
3. The robot gripping method based on deep reinforcement learning according to claim 2, wherein: the hardware layer system comprises a robot, a 3D camera, a computer, a pneumatic clamp and an air compressor.
4. The robot gripping method based on deep reinforcement learning according to claim 2, wherein: the software layer system comprises a calibration module, an image processing module, a point cloud processing module, a communication module and a display module.
5. The robot gripping method based on deep reinforcement learning of claim 4, wherein: the calibration module is used for calibrating internal and external parameters of the camera, lens distortion parameters and the position relation between the camera and the robot coordinate system.
6. The robot gripping method based on deep reinforcement learning of claim 4, wherein: the image processing module is used for acquiring an image, filtering and dividing the acquired image, separating a background image from a workpiece image, dividing a projection grating projected on the surface of the workpiece, matching the same coding area, controlling the time sequence of the projector and the camera, and reconstructing the point cloud in three dimensions.
7. The robot gripping method based on deep reinforcement learning of claim 4, wherein: the point cloud processing module is used for extracting three-dimensional point cloud key points and describing the key points; and establishing a template library, estimating the rotation and translation relation between the template workpiece and the real workpiece by matching key points of the template point cloud with key points of the workpiece point cloud in the real scene, and finally obtaining the robot grabbing pose through coordinate conversion.
8. The robot gripping method based on deep reinforcement learning of claim 4, wherein: the communication module is used for taking the real pose of the workpiece calculated by the point cloud processing module as input, calculating the final picking pose of the tail end clamp of the robot through coordinate transformation, sending planned discrete track points to the robot through WiFi communication, driving the robot to reach the corresponding pose, and finally controlling the pneumatic clamp to be closed to grasp the part through Modbus communication.
9. The robot gripping method based on deep reinforcement learning of claim 4, wherein: the display module is used for real-time display of the acquired image and display of the point cloud processing module.
CN202311164870.8A 2023-09-11 2023-09-11 Robot part grabbing method based on deep reinforcement learning Pending CN117021111A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311164870.8A CN117021111A (en) 2023-09-11 2023-09-11 Robot part grabbing method based on deep reinforcement learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311164870.8A CN117021111A (en) 2023-09-11 2023-09-11 Robot part grabbing method based on deep reinforcement learning

Publications (1)

Publication Number Publication Date
CN117021111A true CN117021111A (en) 2023-11-10

Family

ID=88633867

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311164870.8A Pending CN117021111A (en) 2023-09-11 2023-09-11 Robot part grabbing method based on deep reinforcement learning

Country Status (1)

Country Link
CN (1) CN117021111A (en)

Similar Documents

Publication Publication Date Title
CN108399639B (en) Rapid automatic grabbing and placing method based on deep learning
US20190184570A1 (en) Intelligent robots
US8244402B2 (en) Visual perception system and method for a humanoid robot
Ferreira et al. Stereo-based real-time 6-DoF work tool tracking for robot programing by demonstration
KR20190108191A (en) Deep machine learning methods and apparatus for robotic grasping
CN114097004A (en) Autonomous task performance based on visual embedding
CN111085997A (en) Capturing training method and system based on point cloud acquisition and processing
JP2012218119A (en) Information processing apparatus, method for controlling the same, and program
Husain et al. Realtime tracking and grasping of a moving object from range video
US20230086122A1 (en) Human-Robot Collaborative Flexible Manufacturing System and Method
Zhang et al. Deep learning reactive robotic grasping with a versatile vacuum gripper
Ruan et al. Feature-based autonomous target recognition and grasping of industrial robots
Arents et al. Construction of a smart vision-guided robot system for manipulation in a dynamic environment
Aleotti et al. Trajectory reconstruction with nurbs curves for robot programming by demonstration
Ogawara et al. Acquiring hand-action models in task and behavior levels by a learning robot through observing human demonstrations
Luo et al. A scalable modular architecture of 3D object acquisition for manufacturing automation
CN117021111A (en) Robot part grabbing method based on deep reinforcement learning
CN113436293B (en) Intelligent captured image generation method based on condition generation type countermeasure network
Zhou et al. A cooperative shared control scheme based on intention recognition for flexible assembly manufacturing
CN117377560A (en) Object handling in case of collision avoidance using complementary constraints
Al-Junaid ANN based robotic arm visual servoing nonlinear system
Infantino et al. Visual control of a robotic hand
CN113510718A (en) Intelligent meal selling robot based on machine vision and use method thereof
Sileo et al. Real-time object detection and grasping using background subtraction in an industrial scenario
Durovsky Point cloud based bin picking: object recognition and pose estimation using region growing segmentation algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination