CN116128960A - Automatic workpiece grabbing method, system and device based on machine learning - Google Patents
Automatic workpiece grabbing method, system and device based on machine learning Download PDFInfo
- Publication number
- CN116128960A CN116128960A CN202310253672.2A CN202310253672A CN116128960A CN 116128960 A CN116128960 A CN 116128960A CN 202310253672 A CN202310253672 A CN 202310253672A CN 116128960 A CN116128960 A CN 116128960A
- Authority
- CN
- China
- Prior art keywords
- workpiece
- neural network
- machine learning
- convolutional neural
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000010801 machine learning Methods 0.000 title claims abstract description 25
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 36
- 238000001514 detection method Methods 0.000 claims abstract description 27
- 238000013528 artificial neural network Methods 0.000 claims abstract description 18
- 238000012549 training Methods 0.000 claims abstract description 15
- 238000007781 pre-processing Methods 0.000 claims abstract description 14
- 238000012795 verification Methods 0.000 claims abstract description 5
- 230000006870 function Effects 0.000 claims description 15
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 6
- 230000036544 posture Effects 0.000 claims description 6
- 230000005284 excitation Effects 0.000 claims description 3
- 238000003709 image segmentation Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 230000010354 integration Effects 0.000 claims description 2
- 238000012545 processing Methods 0.000 claims description 2
- 238000010200 validation analysis Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000002790 cross-validation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1612—Programme controls characterised by the hand, wrist, grip control
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mechanical Engineering (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Robotics (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Orthopedic Medicine & Surgery (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of machine learning, image recognition and binocular vision, and discloses a method, a system and a device for automatically grabbing workpieces based on machine learning, aiming at the problems that the existing automatic grabbing of workpieces is complex in system composition, inaccurate in workpiece category recognition, difficult to expand the types of grabbed targets, inefficient, inaccurate and the like in the application process. The device comprises a system and a mechanical arm control assembly, wherein the system comprises a binocular vision assembly, a data preprocessing frame, a deep convolutional neural network and an information flow; according to the method, target detection information output by the deep convolutional neural network is fused with image information of the binocular vision component through an information flow, the complexity of the system is reduced, the data set is randomly extracted by the data preprocessing frame to form a training set and a verification set, the management of the workpiece library types is completed, the recognition of multiple types of workpieces is realized by the deep neural network, the system is more efficient, easier to expand and more accurate to classify and position, and the automatic grabbing task of the workpieces is completed.
Description
The application is a divisional application of a patent application named as an automatic workpiece grabbing system and method based on machine learning, and the application date of the original application is 2021, 09, 17 and 202111095314.0.
Technical Field
The invention belongs to the technical field of machine learning, image recognition and binocular vision, and particularly relates to a workpiece automatic grabbing method, system and device based on machine learning.
Background
The task of the automatic workpiece grabbing technology is to finish the identification of the tools with the specified types, provide depth information and grabbing point information of target workpieces for a mechanical grabbing arm, effectively identify all workpiece types in a workpiece library and manage the identified types as required.
At present, the automatic workpiece grabbing technology mainly relies on means such as target feature extraction and feature matching to classify and position, and meanwhile, the measurement of depth information of a workpiece generally adopts a laser ranging or multi-graph triangulation mode, so that the following defects exist:
firstly, it is: in the partial automatic grabbing technology, an active visual scene measurement method is adopted for the spatial positioning of a workpiece, a visual component realizes the construction of a three-dimensional space by emitting laser outwards, and a point cloud image formed by a laser sensor is utilized to acquire the spatial positioning of the workpiece, so that a system is complex and the hardware requirement is high;
secondly, it is: the target detection of the workpiece adopts a characteristic matching and lightweight neural network mode, so that the effect of target classification and positioning is poor, the type expansion of the identifiable workpiece is difficult, and the multi-target detection capability is weak;
thirdly, the method comprises the following steps: the gripping of workpieces is generally performed by a geometric guiding method, so that the gripping efficiency of the workpieces is low under a dynamic condition.
Disclosure of Invention
Aiming at the problems that the existing automatic workpiece grabbing is complex in system composition, inaccurate in workpiece category identification, difficult to expand the type of the grabbed target, inefficient, inaccurate and the like in the application process, the invention provides an automatic workpiece grabbing method, an automatic workpiece grabbing system and an automatic workpiece grabbing device based on machine learning.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the invention provides a machine learning-based automatic workpiece grabbing method, which comprises the following steps of:
step 1, image acquisition and pretreatment;
collecting multiple gesture images of a workpiece in different environments; obtaining a data set and a label according to the gesture image;
step 2, training a deep convolutional neural network Net 1;
training a deep convolutional neural network by adopting the data set and the label;
obtaining parameters of the deep convolutional neural network by taking the minimum value of the loss function of the deep convolutional neural network as a target;
updating the weight of the deep convolutional neural network by adopting an Adam optimizer to obtain an updated weight;
calculating a loss variation value of the loss function according to the updated weight and the parameter;
if the loss change value is smaller than a set threshold value, taking the depth convolution neural network corresponding to the loss value as a converged depth convolution neural network;
step 3, adopting K-fold to cross and verify the weight of the deep convolutional neural network Net1 obtained by training in the step 2;
step 4, detecting a workpiece target in real time;
inputting the real-time video image of the workpiece into the depth convolution neural network Net1 verified in the step 3, and outputting target detection information;
step 5, performing space positioning on the workpiece;
calculating spatial depth information according to the target detection information;
step 6, completing a grabbing task on the corresponding workpiece;
and grabbing corresponding workpieces in real time according to the space depth information.
Optionally, the specific process of image acquisition in the step 1 is: the binocular vision assembly is provided with a left camera and a right camera, and the left camera is used for collecting images of various postures of a workpiece in different environments.
Optionally, the preprocessing includes:
marking different targets in the acquired gesture images by adopting an image segmentation method to obtain a data set and corresponding labels; the label is workpiece category information; the data set is an image acquired by the type of the workpiece library; the data set includes a training set and a validation set.
Optionally, the input layer of the deep convolutional neural network Net1 is a visible light image, the size of the image is W.H, various types of convolutional kernels are adopted to carry out convolutional operation, the characteristic image is downsampled by adjusting the convolutional step length, the excitation function is a ReLU function, and the target detection is completed under three scales through the characteristic pooling pyramid FPN network and the convolutional layer; the process may map the image to a one-dimensional vector of object detection information.
Optionally, the loss function is:
L c (o,c,O,C,l,g)＝λ 1 L conf (o,c)+λ 2 L cla (O,C)+λ 3 L loc (l,g)
wherein L is conf (o, c) is confidenceLoss of degree; l (L) cla (O, C) is a classification loss; l (L) loc (l, g) is a loss of positioning; lambda (lambda) 1 A balance coefficient that is a confidence loss; lambda (lambda) 2 Balance coefficient for classification loss; lambda (lambda) 3 To locate the balance of the losses.
Optionally, updating the weight of the deep convolutional neural network by adopting an Adam optimizer to obtain an updated weight, which specifically comprises:
m t ＝β 1 ·m t-1 +(1-β 1 )·g(ω t )
ν t ＝β 2 ·ν t-1 +(1-β 2 )·g(ω t )·g(ω t )
wherein m is t To weight first order momentum, m t-1 Is the first order momentum before weight update, beta 1 (0.9)、β 2 (0.999) to control the attenuation coefficient, g (ω) t ) For time t to parameter omega t Is a loss gradient of v t Is the second order momentum, v t-1 Is the second order momentum before updating the network weight, alpha is the learning rate, epsilon (10 -7 ) To prevent the denominator from being zero decimal, ω t Is the weight parameter before updating omega t+1 Is the weight parameter after the update of the weight parameter,
all are calculated intermediate values. />
Optionally, calculating spatial depth information according to the target detection information specifically includes:
and processing by using a correlation matching algorithm and a triangulation principle, and calculating the spatial depth information of the workpiece.
The invention also provides a machine learning-based workpiece automatic grabbing system, which comprises a binocular vision component, a data preprocessing frame, a deep convolutional neural network and an information flow;
the binocular vision assembly is provided with a left high-definition camera and a right high-definition camera and is used for collecting images of a workpiece in various postures under different environments;
the data preprocessing framework is used for:
image data integration is carried out on multiple types of workpieces, and an image data set is established for each type of workpiece by using a manual labeling mode;
randomly extracting the data set to form a training set and a verification set, and managing the class of the workpiece library;
the deep convolutional neural network detects a workpiece target;
the information flow fuses the target detection information obtained by the depth convolution neural network with the image information of the left camera and the right camera of the binocular vision component, and performs space positioning on a workpiece to obtain space depth information; the spatial depth information is used to enable the corresponding workpiece to be grasped in real time.
The invention also provides a machine learning-based automatic workpiece grabbing device, which comprises a mechanical arm control assembly and a machine learning-based automatic workpiece grabbing system;
the mechanical arm control assembly is used for grabbing corresponding workpieces in real time according to the space depth information.
The information flow applies the target detection information obtained by the deep convolutional neural network to the spatial positioning of the binocular vision component on the workpiece, solves the problems that the workpiece type identification is inaccurate, the grabbed target type is difficult to expand, the efficiency is low, the accuracy is poor and the like, fuses the target detection information output by the deep convolutional neural network with the image information of the left camera and the right camera of the binocular vision component through the information flow, successfully reduces the complexity of the workpiece grabbing system, and enables the system to be more suitable for complex conditions.
Compared with the prior art, the invention has the following advantages:
1. the invention provides a machine learning-based workpiece automatic grabbing system, which comprises a binocular vision component, a data preprocessing frame, a deep convolutional neural network and an information flow, wherein the binocular vision component is used for preprocessing data; according to the system, the target detection information output by the deep convolutional neural network is fused with the image information of the left camera and the right camera of the binocular vision component through the information flow, so that the complexity of the workpiece grabbing system is successfully reduced, and the system can be more suitable for complex conditions.
2. The invention provides a machine learning-based automatic workpiece grabbing method, which adopts a method of combining a data preprocessing frame and a deep convolutional neural network to use: the data prediction management framework completes random extraction of the data set to form a training set and a verification set, and simultaneously completes management of the class of the workpiece library, and the deep neural network realizes identification of multiple classes of workpieces. The automatic grabbing task of the workpiece can be completed with higher efficiency, a system which is easier to expand and more accurate classification and positioning.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
A machine learning-based automatic workpiece grabbing system comprises a binocular vision component, a data preprocessing frame, a deep convolutional neural network and an information flow.
The binocular vision assembly is provided with left and right high-definition cameras and is used for collecting images of a workpiece in different environments and in various postures.
The data preprocessing framework firstly integrates image data for multiple types of workpieces, establishes image data sets for various workpieces by using a manual labeling mode, and simultaneously enables the deep neural network to successfully realize accurate identification of the multiple types of workpieces, the data preprocessing framework can complete random extraction of the data sets to form training sets and verification sets, and the framework can complete management of workpiece library types.
And the deep convolution neural network performs workpiece target detection.
The information flow applies the target detection information obtained by the deep convolutional neural network to the spatial positioning of the binocular vision component on the workpiece, and solves the problems that the workpiece type identification is inaccurate, the grabbed target type is difficult to expand, the efficiency is low, the accuracy is low and the like.
The invention is described with reference to fig. 1, in which the process of spatial positioning of a workpiece by a vision component in an automatic workpiece grabbing system is resolved into the detection problem of a workpiece target in a real-time video and the acquisition problem of depth information of the workpiece by a binocular vision component, the real-time target detection is performed on the workpiece in the real-time video of a left camera by constructing a depth convolution neural network, the detection information is transmitted to the binocular vision component through an information flow, and the component combines the target detection information, performs correlation matching and triangulation on the real-time video images of the left and right cameras, and outputs the real-time spatial position of the workpiece to be detected in real time. In particular to the construction of a deep neural network, the optimization of network training and the spatial positioning of workpieces.
(1) A deep neural network for multi-type workpiece recognition is trained, the network is denoted as Net1, and the network type is a convolutional neural network.
(1-1) setting an identification network Net1: the structure of the network is a CNN network, the input layer is a visible light image, the size of the image is W.H, convolution operation is carried out by adopting various convolution kernels, the characteristic image is downsampled by adjusting the convolution step length, the excitation function is a ReLU function, the target detection is completed under three scales by the characteristic pooling pyramid FPN network and the convolution layer, and the physical meaning of the process is that the image is mapped into a target detection information one-dimensional vector.
(1-2) the binocular vision component is provided with a left camera and a right camera, the left camera is used for collecting images of various postures of a workpiece in different environments, and different targets are marked by adopting an image segmentation method. Resulting in a dataset d= { D 1 ,d 2 ,...d n Corresponding tag s= { [ S } 11 ,s 12 ...],[s 21 ,s 22 ...],...[s n1 ,s n2 ...]}。
(1-3) defining a Net1 deep convolutional neural network loss function of L for real classification and localization of workpiece targets c (o,c,O,C,l,g)＝λ 1 L conf (o,c)+λ 2 L cla (O,C)+λ 3 L loc (l, g) wherein the three components are confidence, classification and location loss, lambda, respectively 1 、λ 2 、λ 3 Is the balance coefficient between the three partial losses. Training convergence to deep neural networks to L c For the purpose of minimization, parameters that optimize the convolutional neural network (minimizing the gradient of the loss function to the convolutional neural network parameters) are calculated.
(1-4) updating the weight of the convolutional neural network by adopting an Adam optimizer, wherein the Adam optimizer can adapt to the learning rate, and the parameter updating effect is further improved:
m t ＝β 1 ·m t-1 +(1-β 1 )·g(ω t )
ν t ＝β 2 ·ν t-1 +(1-β 2 )·g(ω t )·g(ω t )
wherein m is t To weight first order momentum, m t-1 Is the first order momentum before weight update, beta 1 (0.9)、β 2 (0.999) to control the attenuation coefficient, g (ω) t ) For time t to parameter omega t Is a loss gradient of v t Is the second order momentum, v t-1 Is the second order momentum before updating the network weight, alpha is the learning rate, epsilon (10 -7 ) To prevent the denominator from being zero decimal, ω t Is the weight parameter before updating omega t+1 Is the weight parameter after the update of the weight parameter,
all are calculated intermediate values.
(1-5) repeating steps 1-3 to 1-4, wherein the loss function variation is smaller than the threshold eta after N continuous optimizations, and the Net1 network converges.
(2) And optimizing the weight of the iterative deep neural network by adopting a K-fold cross validation strategy.
(3) Under the dynamic condition, inputting a real-time video image acquired by a left camera of the binocular vision component into a depth convolutional neural network Net1, detecting a real-time target of the environment where the workpiece is located, and outputting target detection information.
(4) The binocular vision component acquires target detection information of a workpiece to be grabbed through an information transmission flow, and image information acquired by the left camera and the right camera of the binocular vision component is processed by utilizing a correlation matching algorithm and a triangulation principle to calculate the spatial depth information of the workpiece.
(5) And in a state of real-time detection, the space depth information of the workpiece is transmitted to the mechanical arm control assembly through an information transmission flow, so that real-time grabbing of the corresponding workpiece is completed.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (9)
1. The automatic workpiece grabbing method based on machine learning is characterized by comprising the following steps of:
step 1, image acquisition and pretreatment;
collecting multiple gesture images of a workpiece in different environments; obtaining a data set and a label according to the gesture image;
step 2, training a deep convolutional neural network Net 1;
training a deep convolutional neural network by adopting the data set and the label;
obtaining parameters of the deep convolutional neural network by taking the minimum value of the loss function of the deep convolutional neural network as a target;
updating the weight of the deep convolutional neural network by adopting an Adam optimizer to obtain an updated weight;
calculating a loss variation value of the loss function according to the updated weight and the parameter;
if the loss change value is smaller than a set threshold value, taking the depth convolution neural network corresponding to the loss value as a converged depth convolution neural network;
step 3, adopting K-fold to cross and verify the weight of the deep convolutional neural network Net1 obtained by training in the step 2;
step 4, detecting a workpiece target in real time;
inputting the real-time video image of the workpiece into the depth convolution neural network Net1 verified in the step 3, and outputting target detection information;
step 5, performing space positioning on the workpiece;
calculating spatial depth information according to the target detection information;
step 6, completing a grabbing task on the corresponding workpiece;
and grabbing corresponding workpieces in real time according to the space depth information.
2. The automatic workpiece grabbing method based on machine learning according to claim 1, wherein the specific process of image acquisition in step 1 is as follows: the binocular vision assembly is provided with a left camera and a right camera, and the left camera is used for collecting images of various postures of a workpiece in different environments.
3. The machine learning based automatic workpiece gripping method of claim 1, wherein the preprocessing comprises:
marking different targets in the acquired gesture images by adopting an image segmentation method to obtain a data set and corresponding labels; the label is workpiece category information; the data set is an image acquired by the type of the workpiece library; the data set includes a training set and a validation set.
4. The automatic workpiece grabbing method based on machine learning according to claim 1, wherein an input layer of the deep convolutional neural network Net1 is a visible light image, the size of the image is w×h, convolution operation is performed by adopting a plurality of types of convolution kernels, feature images are downsampled by adjusting convolution step sizes, an excitation function is a ReLU function, and target detection is completed under three scales through a feature pooling pyramid FPN network and a convolution layer.
5. The machine learning based automatic workpiece gripping method of claim 1, wherein the loss function is:
L c (o,c,O,C,l,g)＝λ 1 L conf (o,c)+λ 2 L cla (O,C)+λ 3 L loc (l,g)
wherein L is conf (o, c) is a confidence loss; l (L) cla (O, C) is a classification loss; l (L) loc (l, g) is a loss of positioning; lambda (lambda) 1 A balance coefficient that is a confidence loss; lambda (lambda) 2 Balance coefficient for classification loss; lambda (lambda) 3 To locate the balance of the losses.
6. The machine learning based automatic workpiece grabbing method of claim 1, wherein the weight of the deep convolutional neural network is updated by an Adam optimizer to obtain an updated weight, and the method specifically comprises:
m t ＝β 1 ·m t-1 +(1-β 1 )·g(ω t )
ν t ＝β 2 ·ν t-1 +(1-β 2 )·g(ω t )·g(ω t )
wherein m is t To weight first order momentum, m t-1 Is the first order momentum before weight update, beta 1 (0.9)、β 2 (0.999) to control the attenuation coefficient, g (ω) t ) For time t to parameter omega t Is a loss gradient of v t Is the second order momentum, v t-1 Is the second order momentum before updating the network weight, alpha is the learning rate, epsilon (10 -7 ) To prevent the denominator from being zero decimal, ω t Is the weight parameter before updating omega t+1 Is the weight parameter after the update of the weight parameter,
all are calculated intermediate values.
7. The machine learning based automatic workpiece gripping method according to claim 1, wherein the calculating of the spatial depth information based on the target detection information specifically includes:
and processing by using a correlation matching algorithm and a triangulation principle, and calculating the spatial depth information of the workpiece.
8. The automatic workpiece grabbing system based on machine learning is characterized by comprising a binocular vision component, a data preprocessing frame, a deep convolutional neural network and an information flow;
the binocular vision assembly is provided with a left high-definition camera and a right high-definition camera and is used for collecting images of a workpiece in various postures under different environments;
the data preprocessing framework is used for:
image data integration is carried out on multiple types of workpieces, and an image data set is established for each type of workpiece by using a manual labeling mode;
randomly extracting the data set to form a training set and a verification set, and managing the class of the workpiece library;
the deep convolutional neural network detects a workpiece target;
the information flow fuses the target detection information output by the depth convolution neural network with the image information of the left camera and the right camera of the binocular vision component, and performs space positioning on a workpiece to obtain space depth information; the spatial depth information is used to enable the corresponding workpiece to be grasped in real time.
9. A machine learning based automatic workpiece gripping apparatus, comprising a robotic arm control assembly and the machine learning based automatic workpiece gripping system of claim 8;
the mechanical arm control assembly is used for grabbing corresponding workpieces in real time according to the space depth information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310253672.2A CN116128960A (en) | 2021-09-17 | 2021-09-17 | Automatic workpiece grabbing method, system and device based on machine learning |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310253672.2A CN116128960A (en) | 2021-09-17 | 2021-09-17 | Automatic workpiece grabbing method, system and device based on machine learning |
| CN202111095314.0A CN113808197A (en) | 2021-09-17 | 2021-09-17 | Automatic workpiece grabbing system and method based on machine learning |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111095314.0A Division CN113808197A (en) | 2021-09-17 | 2021-09-17 | Automatic workpiece grabbing system and method based on machine learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116128960A true CN116128960A (en) | 2023-05-16 |
Family
ID=78895844
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111095314.0A Pending CN113808197A (en) | 2021-09-17 | 2021-09-17 | Automatic workpiece grabbing system and method based on machine learning |
| CN202310253672.2A Withdrawn CN116128960A (en) | 2021-09-17 | 2021-09-17 | Automatic workpiece grabbing method, system and device based on machine learning |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111095314.0A Pending CN113808197A (en) | 2021-09-17 | 2021-09-17 | Automatic workpiece grabbing system and method based on machine learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN113808197A (en) |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108161931A (en) * | 2016-12-07 | 2018-06-15 | 广州映博智能科技有限公司 | The workpiece automatic identification of view-based access control model and intelligent grabbing system |
| CN106874914B (en) * | 2017-01-12 | 2019-05-14 | 华南理工大学 | A kind of industrial machinery arm visual spatial attention method based on depth convolutional neural networks |
| CN108171748B (en) * | 2018-01-23 | 2021-12-07 | 哈工大机器人（合肥）国际创新研究院 | Visual identification and positioning method for intelligent robot grabbing application |
| CN109344772B (en) * | 2018-09-30 | 2021-01-26 | 中国人民解放军战略支援部队信息工程大学 | Ultrashort wave specific signal reconnaissance method based on spectrogram and deep convolutional network |
| CN111368852A (en) * | 2018-12-26 | 2020-07-03 | 沈阳新松机器人自动化股份有限公司 | Article identification and pre-sorting system and method based on deep learning and robot |
| CN111768449B (en) * | 2019-03-30 | 2024-05-14 | 北京伟景智能科技有限公司 | Object grabbing method combining binocular vision with deep learning |
| CN110322515A (en) * | 2019-07-02 | 2019-10-11 | 工极智能科技（苏州）有限公司 | Workpiece identification and grabbing point extraction method based on binocular stereo vision |
| CN110363253A (en) * | 2019-07-25 | 2019-10-22 | 安徽工业大学 | A kind of Surfaces of Hot Rolled Strip defect classification method based on convolutional neural networks |
| CN110728223A (en) * | 2019-10-08 | 2020-01-24 | 济南东朔微电子有限公司 | Helmet wearing identification method based on deep learning |
| CN110969158B (en) * | 2019-11-06 | 2023-07-25 | 中国科学院自动化研究所 | Target detection method, system and device based on underwater operation robot vision |
| CN110942142B (en) * | 2019-11-29 | 2021-09-17 | 广州市百果园信息技术有限公司 | Neural network training and face detection method, device, equipment and storage medium |
| CN111340132B (en) * | 2020-03-10 | 2024-02-02 | 南京工业大学 | Machine olfaction mode identification method based on DA-SVM |
| CN111723782A (en) * | 2020-07-28 | 2020-09-29 | 北京印刷学院 | Deep learning-based visual robot grabbing method and system |
| CN113052005B (en) * | 2021-02-08 | 2024-02-02 | 湖南工业大学 | Garbage sorting method and garbage sorting device for household service |
| CN113222940B (en) * | 2021-05-17 | 2022-07-12 | 哈尔滨工业大学 | Method for automatically grabbing workpiece by robot based on RGB-D image and CAD model |
-
2021
- 2021-09-17 CN CN202111095314.0A patent/CN113808197A/en active Pending
- 2021-09-17 CN CN202310253672.2A patent/CN116128960A/en not_active Withdrawn
Also Published As
| Publication number | Publication date |
|---|---|
| CN113808197A (en) | 2021-12-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109685141B (en) | Robot article sorting visual detection method based on deep neural network | |
| CN107833236B (en) | Visual positioning system and method combining semantics under dynamic environment | |
| CN109559320B (en) | Method and system for realizing visual SLAM semantic mapping function based on hole convolution deep neural network | |
| CN108520203B (en) | Multi-target feature extraction method based on fusion of self-adaptive multi-peripheral frame and cross pooling feature | |
| CN110287907B (en) | Object detection method and device | |
| CN114693661A (en) | Rapid sorting method based on deep learning | |
| CN110310305B (en) | Target tracking method and device based on BSSD detection and Kalman filtering | |
| CN111027555B (en) | License plate recognition method and device and electronic equipment | |
| CN110827321B (en) | Multi-camera collaborative active target tracking method based on three-dimensional information | |
| CN107025657A (en) | A kind of vehicle action trail detection method based on video image | |
| CN114677323A (en) | Semantic vision SLAM positioning method based on target detection in indoor dynamic scene | |
| CN110838145A (en) | Visual positioning and mapping method for indoor dynamic scene | |
| JP2024528419A (en) | Method and apparatus for updating an object detection model | |
| CN105976397A (en) | Target tracking method based on half nonnegative optimization integration learning | |
| CN113657423A (en) | Target detection method suitable for small-volume parts and stacked parts and application thereof | |
| CN110147724B (en) | Method, apparatus, device, and medium for detecting text region in video | |
| CN109544632B (en) | Semantic SLAM object association method based on hierarchical topic model | |
| CN114863201A (en) | Training method and device of three-dimensional detection model, computer equipment and storage medium | |
| CN113139432A (en) | Industrial packaging behavior identification method based on human body skeleton and local image | |
| CN116105721B (en) | Loop optimization method, device and equipment for map construction and storage medium | |
| CN117237411A (en) | Pedestrian multi-target tracking method based on deep learning | |
| CN116128960A (en) | Automatic workpiece grabbing method, system and device based on machine learning | |
| CN115205806A (en) | Method and device for generating target detection model and automatic driving vehicle | |
| CN115273219A (en) | Yoga action evaluation method and system, storage medium and electronic equipment | |
| CN112200850A (en) | ORB extraction method based on mature characteristic points |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20230516 |