CN109784297A - A kind of Three-dimensional target recognition based on deep learning and Optimal Grasp method - Google Patents
A kind of Three-dimensional target recognition based on deep learning and Optimal Grasp method Download PDFInfo
- Publication number
- CN109784297A CN109784297A CN201910077632.0A CN201910077632A CN109784297A CN 109784297 A CN109784297 A CN 109784297A CN 201910077632 A CN201910077632 A CN 201910077632A CN 109784297 A CN109784297 A CN 109784297A
- Authority
- CN
- China
- Prior art keywords
- target object
- image
- convolutional layer
- optimal grasp
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The present invention relates to a kind of Three-dimensional target recognitions based on deep learning and Optimal Grasp method.This method, firstly, obtaining image using Kinect camera；Then, an improved faster RCNN model is constructed in the first order, the target object in image is identified and positioned, the region being partitioned into where target object is simultaneously rotated accordingly；Finally, constructing a new faster RCNN model in the second level, the coordinate and rotation angle at target object Optimal Grasp position are obtained, realizes the Optimal Grasp of target object.The method of the present invention improves model in target identification portion, strengthens identifying and positioning for the target object small to imaging, and using the posture for determining target object based on the method for dichotomy, reduces runing time, improve precision；This method first detects entire target object, then out of detection region searching object optimal pose, not only reduce find feature range, also reduce identification error probability.
Description
Technical field
The present invention relates to technical field of robot vision, and in particular to a kind of Three-dimensional target recognition based on deep learning with
Optimal Grasp method.
Background technique
With the fast development of robot and the application of machine vision, robot crawl also produces many variations.From original
Come the simple intelligent recognition grabbed till now, Optimal Grasp and makes a response to external environment.These variations all signify machine
Device people is just gradually developed to intelligent direction, so that the various actions of robot are increasingly as the mankind.
In field of machine vision, target detection and Optimal Grasp are all research hotspots in recent years.Optimal Grasp requires machine
Device portrait people is the same, can not only identify the classification of target object, also to find the pose that target object is easier crawl.It passes
The crawl pose method of system handles image information using traditional feature extracting method, these methods are generally by designer
It is formed for particular problem hand-designed, because by factors such as the shape of target object, size, angle change, exterior light photographs
It influences, thus extracted feature is extensive indifferent, robustness is poor, it is difficult to adapt to new object.Feature is extracted with tradition
Method is compared, and the advantage of deep learning is not needing artificially to carry out setting certain feature when extracting feature, but uses
A kind of general learning process makes model learn the feature of target object from large-scale data.Therefore depth learning technology is answered
For in the target detection and Optimal Grasp of robot.
At present in the robot Optimal Grasp field based on deep learning, target object may be implemented there are many method
Optimal Grasp, but be difficult to realize that the Optimal Grasp of lesser target object is imaged, and it is longer to handle the time.
Summary of the invention
The purpose of the present invention is to provide a kind of Three-dimensional target recognitions based on deep learning and Optimal Grasp method, solve
It is difficult to realize that the Optimal Grasp of lesser target object and processing time longer problem is imaged.
To achieve the above object, the technical scheme is that a kind of Three-dimensional target recognition based on deep learning with most
Excellent grasping means, includes the following steps:
Step S1, image is obtained using Kinect camera, and carries out image preprocessing；
Step S2, an improved faster RCNN model is constructed in the first order, the target object in image is known
It is not rotated accordingly simultaneously with positioning, the region being partitioned into where target object；
Step S3, a new faster RCNN model is constructed in the second level, obtains target object Optimal Grasp position
Coordinate and rotation angle, realize the Optimal Grasp of target object.
Further, in step sl, described image pre-processes, including carries out contours extract to depth image, and will wheel
Wide image, color image and depth image carry out pixel value by the weight of predetermined ratio respectively and are added, and obtain blending image.
Further, in step s 2, the improved faster RCNN model, including sequentially connected convolutional layer 1,
Pond layer 1, convolutional layer 2, convolutional layer 3, convolutional layer 4, pond layer 2, convolutional layer 5, fused layer, the convolutional layer 3 is gone back and fused layer
Connection, meanwhile, in order to the small object of recognition imaging, the parameter of convolutional layer 2 and convolutional layer 4 is adjusted, i.e. convolutional layer 2
Parameter adjustment are as follows: convolution kernel size is 3*3, and border extended number is 1, step-length 1；The parameter of convolutional layer 4 adjusts are as follows: convolution kernel is big
Small is 5*5, and border extended number is 2, step-length 2
Further, in step s 2, described to construct an improved faster RCNN model in the first order, to image
In the specific implementation process that is identified and positioned of target object are as follows: the fusion that image preprocessing obtains will be carried out through step S1
Image inputs improved faster RCNN model, merges by the extraction of 5 layers of convolutional layer and characteristic image, then with train
Category feature be compared, obtain classification and the position of target object.
Further, in step s 2, the region being partitioned into where target object and the specific mistake rotated accordingly
Journey are as follows:
Step S21, identifying and positioning for the target object in image is found according to improved faster RCNN model
The profile of target object, so that target object be split from image:
Step S22, the profile of target object is surrounded using minimum area-encasing rectangle, and determines rotating range:
Step S23, image is carried out by multiple rotary using dichotomy.
Compared to the prior art, the invention has the following advantages: the present invention uses tandem type faster RCNN model
Realize the Optimal Grasp of target object.In the improved faster RCNN model of the first order, the spy of target object is saved as far as possible
Sign realizes identification and positioning to small target object is imaged.It is accurate using dichotomy in the faster RCNN model of the second level
It determines the posture of target object, there is the imaging object suitable for arbitrary size, improve position and attitude accuracy and reduction processing
The features such as time.Automation industrial to realization, intelligence have good application prospect.
Detailed description of the invention
Fig. 1 is the improved faster RCNN model schematic of the present invention.
Fig. 2 is the flow chart of robot Optimal Grasp system.
Specific embodiment
With reference to the accompanying drawing, technical solution of the present invention is specifically described.
The present invention provides a kind of Three-dimensional target recognitions based on deep learning and Optimal Grasp method, including walk as follows
It is rapid:
Step S1, image is obtained using Kinect camera, and carries out image preprocessing；
Step S2, an improved faster RCNN model is constructed in the first order, the target object in image is known
It is not rotated accordingly simultaneously with positioning, the region being partitioned into where target object；
Step S3, a new faster RCNN model is constructed in the second level, obtains target object Optimal Grasp position
Coordinate and rotation angle, realize the Optimal Grasp of target object.
In step s 2, the improved faster RCNN model, including sequentially connected convolutional layer 1, pond layer 1, volume
Lamination 2, convolutional layer 3, convolutional layer 4, pond layer 2, convolutional layer 5, fused layer, the convolutional layer 3 are also connect with fused layer, meanwhile,
In order to the small object of recognition imaging, the parameter of convolutional layer 2 and convolutional layer 4 is adjusted.Before improvement, the ginseng of convolutional layer 2
Number are as follows: convolution kernel size is 5*5, and border extended number is 2, step-length 2；The parameter of convolutional layer 4: convolution kernel size is 3*3, edge
Spreading number is 1, step-length 1.After improvement, the parameter of convolutional layer 2 are as follows: convolution kernel size is 3*3, and border extended number is 1, and step-length is
1；The parameter of convolutional layer 4: convolution kernel size is 5*5, and border extended number is 2, step-length 2.
The following are specific implementation processes of the invention.
As shown in Fig. 2, a kind of Three-dimensional target recognition based on deep learning of the invention and Optimal Grasp method, including such as
Lower step:
S1, image is obtained using Kinect camera, and is pre-processed；
S2, it is input to the improved faster RCNN model of the first order, detects classification and the position of target object；
S3, segmented image are simultaneously rotated；
S4, it is input to second level faster RCNN model, obtains position and the posture of target object Optimal Grasp.
In S1 when image preprocessing, contours extract is carried out to depth image, and by contour images, color image and depth
The weight of image respectively by a certain percentage carries out pixel value addition, obtains blending image.
It is to input blending image obtained in S1 when being input to the improved faster RCNN model of the first order in S2
Into improved faster RCNN model, merged by the extraction of 5 layers of feature extraction layer and characteristic image, then with train
Category feature be compared, obtain classification and the position of target object.
It is to be split according to the rectangle frame of first order faster RCNN model output, by mesh in S3 when segmented image
Mark object is split from overall background.When rotating image in S3, target object frame is risen using minimum area-encasing rectangle first
Come.Assuming that lesser angle is α ° between the short side and x-axis of minimum rectangle, β ° of reselection rotation steps appropriate then can be true
Determining rotating range is (alpha-beta) ° and (alpha+beta) °.The image after (alpha-beta) ° and (alpha+beta) ° will be rotated and be input to second level faster
In RCNN model, model can export the probability that rectangle frame in this two images is Optimal Grasp pose.Compare (alpha-beta) ° and (α+
Output probability β) ° chooses probability big one (assuming that being (alpha+beta) °) with α ° and forms new rotating range, then uses two points
Method chooses α ° and the angle (alpha+beta/2) ° of (alpha+beta) ° centre, the image of rotation alpha ° and (alpha+beta) ° is input to trained mould
In type, compare the α ° of output probability with (alpha+beta) °, choose big one of probability and form new rotating range again with (alpha+beta/2) °,
And so on.As long as being computed probably 6 times (obtaining 6 postrotational images) of rotation, so that it may propose attitude angle precision
Height is to 1 °.
When obtaining position and the posture of target object Optimal Grasp in S4, which is in S3 in rotated image
The coordinate of Optimal Grasp needs to be translated into the original image obtained in S1.The posture of target object is rotated according in S3
Angle calculation obtain.
The above are preferred embodiments of the present invention, all any changes made according to the technical solution of the present invention, and generated function is made
When with range without departing from technical solution of the present invention, all belong to the scope of protection of the present invention.
Claims (5)
1. a kind of Three-dimensional target recognition based on deep learning and Optimal Grasp method, which comprises the steps of:
Step S1, image is obtained using Kinect camera, and carries out image preprocessing；
Step S2, the first order construct an improved faster RCNN model, to the target object in image carry out identification and
Positioning, the region being partitioned into where target object are simultaneously rotated accordingly；
Step S3, a new faster RCNN model is constructed in the second level, obtains the coordinate at target object Optimal Grasp position
With rotation angle, the Optimal Grasp of target object is realized.
2. a kind of Three-dimensional target recognition based on deep learning according to claim 1 and Optimal Grasp method, feature
It is, in step sl, described image pretreatment, including contours extract is carried out to depth image, and by contour images, cromogram
Picture carries out pixel value by the weight of predetermined ratio respectively with depth image and is added, and obtains blending image.
3. a kind of Three-dimensional target recognition based on deep learning according to claim 1 and Optimal Grasp method, feature
It is, in step s 2, the improved faster RCNN model, including sequentially connected convolutional layer 1, pond layer 1, convolution
Layer 2, convolutional layer 3, convolutional layer 4, pond layer 2, convolutional layer 5, fused layer, the convolutional layer 3 are also connect with fused layer, meanwhile, it is
Can the small object of recognition imaging, the parameter of convolutional layer 2 and convolutional layer 4 is adjusted, i.e. the parameter adjustment of convolutional layer 2 are as follows:
Convolution kernel size is 3*3, and border extended number is 1, step-length 1；The parameter of convolutional layer 4 adjusts are as follows: convolution kernel size is 5*5, side
Edge spreading number is 2, step-length 2.
4. a kind of Three-dimensional target recognition based on deep learning according to claim 3 and Optimal Grasp method, feature
It is, it is in step s 2, described to construct an improved faster RCNN model in the first order, to the target object in image
The specific implementation process identified and positioned are as follows: the blending image that image preprocessing obtains will be carried out through step S1, input changes
Into faster RCNN model, merged by the extraction of 5 layers of convolutional layer and characteristic image, then with trained category feature
It is compared, obtains classification and the position of target object.
5. a kind of Three-dimensional target recognition based on deep learning according to claim 1 and Optimal Grasp method, feature
It is, in step s 2, the region being partitioned into where target object and the detailed process rotated accordingly are as follows:
Step S21, target is found to identifying and positioning for the target object in image according to improved faster RCNN model
The profile of object, so that target object be split from image:
Step S22, the profile of target object is surrounded using minimum area-encasing rectangle, and determines rotating range:
Step S23, image is carried out by multiple rotary using dichotomy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910077632.0A CN109784297A (en) | 2019-01-26 | 2019-01-26 | A kind of Three-dimensional target recognition based on deep learning and Optimal Grasp method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910077632.0A CN109784297A (en) | 2019-01-26 | 2019-01-26 | A kind of Three-dimensional target recognition based on deep learning and Optimal Grasp method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109784297A true CN109784297A (en) | 2019-05-21 |
Family
ID=66502737
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910077632.0A Pending CN109784297A (en) | 2019-01-26 | 2019-01-26 | A kind of Three-dimensional target recognition based on deep learning and Optimal Grasp method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109784297A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110349138A (en) * | 2019-06-28 | 2019-10-18 | 歌尔股份有限公司 | The detection method and device of the target object of Case-based Reasoning segmentation framework |
| CN111145257A (en) * | 2019-12-27 | 2020-05-12 | 深圳市越疆科技有限公司 | Article grabbing method and system and article grabbing robot |
| CN113657551A (en) * | 2021-09-01 | 2021-11-16 | 陕西工业职业技术学院 | Robot grabbing posture task planning method for sorting and stacking multiple targets |
| CN113838144A (en) * | 2021-09-14 | 2021-12-24 | 杭州印鸽科技有限公司 | Method for positioning object on UV printer based on machine vision and deep learning |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105619413A (en) * | 2016-04-01 | 2016-06-01 | 芜湖哈特机器人产业技术研究院有限公司 | Automatic grabbing device for inner-ring workpieces and control method of automatic grabbing device |
| CN108010078A (en) * | 2017-11-29 | 2018-05-08 | 中国科学技术大学 | A kind of grasping body detection method based on three-level convolutional neural networks |
| CN108510062A (en) * | 2018-03-29 | 2018-09-07 | 东南大学 | A kind of robot irregular object crawl pose rapid detection method based on concatenated convolutional neural network |
| CN108648233A (en) * | 2018-03-24 | 2018-10-12 | 北京工业大学 | A kind of target identification based on deep learning and crawl localization method |
-
2019
- 2019-01-26 CN CN201910077632.0A patent/CN109784297A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105619413A (en) * | 2016-04-01 | 2016-06-01 | 芜湖哈特机器人产业技术研究院有限公司 | Automatic grabbing device for inner-ring workpieces and control method of automatic grabbing device |
| CN108010078A (en) * | 2017-11-29 | 2018-05-08 | 中国科学技术大学 | A kind of grasping body detection method based on three-level convolutional neural networks |
| CN108648233A (en) * | 2018-03-24 | 2018-10-12 | 北京工业大学 | A kind of target identification based on deep learning and crawl localization method |
| CN108510062A (en) * | 2018-03-29 | 2018-09-07 | 东南大学 | A kind of robot irregular object crawl pose rapid detection method based on concatenated convolutional neural network |
Non-Patent Citations (2)
| Title |
|---|
| KEVIN LAI.ET AL: ""A Large-Scale Hierarchical Multi-View RGB-D Object Dataset"", 《IEEE》 * |
| QINGQUAN LIN, DAN CHEN: ""Target Recognition and Optimal Grasping Based on Deep Learning"", 《IEEE》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110349138A (en) * | 2019-06-28 | 2019-10-18 | 歌尔股份有限公司 | The detection method and device of the target object of Case-based Reasoning segmentation framework |
| CN110349138B (en) * | 2019-06-28 | 2021-07-27 | 歌尔股份有限公司 | Target object detection method and device based on example segmentation framework |
| CN111145257A (en) * | 2019-12-27 | 2020-05-12 | 深圳市越疆科技有限公司 | Article grabbing method and system and article grabbing robot |
| CN111145257B (en) * | 2019-12-27 | 2024-01-05 | 深圳市越疆科技有限公司 | Article grabbing method and system and article grabbing robot |
| CN113657551A (en) * | 2021-09-01 | 2021-11-16 | 陕西工业职业技术学院 | Robot grabbing posture task planning method for sorting and stacking multiple targets |
| CN113657551B (en) * | 2021-09-01 | 2023-10-20 | 陕西工业职业技术学院 | Robot grabbing gesture task planning method for sorting and stacking multiple targets |
| CN113838144A (en) * | 2021-09-14 | 2021-12-24 | 杭州印鸽科技有限公司 | Method for positioning object on UV printer based on machine vision and deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109410321B (en) | Three-dimensional reconstruction method based on convolutional neural network | |
| CN109344701B (en) | Kinect-based dynamic gesture recognition method | |
| CN109784297A (en) | A kind of Three-dimensional target recognition based on deep learning and Optimal Grasp method | |
| CN112991447B (en) | Visual positioning and static map construction method and system in dynamic environment | |
| CN110827398B (en) | Automatic semantic segmentation method for indoor three-dimensional point cloud based on deep neural network | |
| CN108648240A (en) | Based on a non-overlapping visual field camera posture scaling method for cloud characteristics map registration | |
| CN113065546B (en) | Target pose estimation method and system based on attention mechanism and Hough voting | |
| CN111553949B (en) | Positioning and grabbing method for irregular workpiece based on single-frame RGB-D image deep learning | |
| CN111209915A (en) | Three-dimensional image synchronous identification and segmentation method based on deep learning | |
| CN108537848A (en) | A kind of two-stage pose optimal estimating method rebuild towards indoor scene | |
| CN107369204B (en) | Method for recovering basic three-dimensional structure of scene from single photo | |
| CN109740537B (en) | Method and system for accurately marking attributes of pedestrian images in crowd video images | |
| CN111368759B (en) | Monocular vision-based mobile robot semantic map construction system | |
| CN110135277B (en) | Human behavior recognition method based on convolutional neural network | |
| CN112163588A (en) | Intelligent evolution-based heterogeneous image target detection method, storage medium and equipment | |
| CN113011288A (en) | Mask RCNN algorithm-based remote sensing building detection method | |
| CN112766136A (en) | Space parking space detection method based on deep learning | |
| CN111709317B (en) | Pedestrian re-identification method based on multi-scale features under saliency model | |
| Zou et al. | Microarray camera image segmentation with Faster-RCNN | |
| CN114898041A (en) | Improved ICP method based on luminosity error | |
| CN116310098A (en) | Multi-view three-dimensional reconstruction method based on attention mechanism and variable convolution depth network | |
| CN113011438B (en) | Bimodal image significance detection method based on node classification and sparse graph learning | |
| CN112669452B (en) | Object positioning method based on convolutional neural network multi-branch structure | |
| CN110516527B (en) | Visual SLAM loop detection improvement method based on instance segmentation | |
| CN117351078A (en) | Target size and 6D gesture estimation method based on shape priori |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190521 |
|
| RJ01 | Rejection of invention patent application after publication |