CN114723827A - Grabbing robot target positioning system based on deep learning - Google Patents
Grabbing robot target positioning system based on deep learning Download PDFInfo
- Publication number
- CN114723827A CN114723827A CN202210460922.5A CN202210460922A CN114723827A CN 114723827 A CN114723827 A CN 114723827A CN 202210460922 A CN202210460922 A CN 202210460922A CN 114723827 A CN114723827 A CN 114723827A
- Authority
- CN
- China
- Prior art keywords
- detection
- image
- positioning system
- data processing
- processing module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013135 deep learning Methods 0.000 title claims abstract description 8
- 238000001514 detection method Methods 0.000 claims abstract description 44
- 238000005259 measurement Methods 0.000 claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 10
- 238000005070 sampling Methods 0.000 claims abstract description 8
- 239000000523 sample Substances 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 7
- 238000000691 measurement method Methods 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003754 machining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Abstract
The invention discloses a grabbing robot target positioning system based on deep learning, and relates to the technical field of robots; the image acquisition module is connected with the data processing module through a USB3.0 data line, the data processing module is connected with the control module through a TCP protocol, a target detection algorithm is adopted as a Yolo target detection algorithm, a Yolov5 network image enters prediction after passing through a backhaul, at the moment, the network performs up-sampling, the image is sequentially decomposed into 13 × 13, 26 × 26 and 52 × 52 parts, and three detection heads respectively perform detection of three sizes, namely large, medium and small; for the detection of small objects, an upper sampling layer is added, an anchor point is set, and a detection probe is added; the invention carries out the detection of smaller articles through the improved scheme, is convenient for quick measurement and improves the accuracy; the grabbing robot target positioning system with higher precision and certain robustness is realized.
Description
Technical Field
The invention belongs to the technical field of robots, and particularly relates to a grabbing robot target positioning system based on deep learning.
Background
The three-dimensional measurement technology is a visualization process of measuring three-dimensional coordinates of points on the surface of an object in a specified coordinate system and storing data through a computer. Three-dimensional measurement techniques have developed dramatically and gradually into commercialization in recent years. In the industrial field, product component machining and assembly often requires precise measurement and modeling of the components. For example, in the aircraft manufacturing industry, the mode of inspecting the product quality by using the traditional two-dimensional pattern and simulation means cannot meet the manufacturing requirements of the novel aircraft.
The existing target detection method is shown in fig. 1; existing three-dimensional measurement techniques can be broadly classified into contact measurement and non-contact measurement according to whether or not contact is made with an object to be measured during measurement. For non-contact measurement, the measurement principle can be divided into acoustic measurement, electromagnetic measurement and optical measurement. The most common acoustic three-dimensional measurement technique is a method of measuring by using ultrasonic waves. For electromagnetic measurement, the electromagnetic wave band adopted in the measurement is divided, and there are mainly CT, microwave radar and other measurement methods.
As shown in fig. 2, which is a three-dimensional measurement method, among all three-dimensional measurement methods, the optical-based three-dimensional measurement technology has great advantages in speed and precision, and is different from lidar, ultrasonic and infrared sensors, and the optical camera has low hardware cost, low trial threshold, small size, and is easy to be used in combination with other sensors. More importantly, on the background of increasingly skilled capability and technology of processing mass information by a computer and under the increasingly high-level information requirements of academia and industry, the camera can capture more information at lower cost, so that the camera stands out of various sensors and becomes the highest choice in cost performance.
The existing target detection algorithm and the three-dimensional measurement method have the phenomena of inaccurate detection, imperfect hardware, low working efficiency and poor stability.
Disclosure of Invention
To solve the problems of the background art; the invention aims to provide a grabbing robot target positioning system based on deep learning.
The invention relates to a grabbing robot target positioning system based on deep learning, which comprises an image acquisition module, a data processing module and a control module, wherein the image acquisition module is used for acquiring an image; the image acquisition module is connected with the data processing module through a USB3.0 data line, the data processing module is connected with the control module through a TCP protocol, a target detection algorithm is adopted as a Yolo target detection algorithm, a Yolov5 network image enters prediction after passing through a backhaul, at the moment, the network performs up-sampling, the image is sequentially decomposed into 13 × 13, 26 × 26 and 52 × 52 parts, and three detection heads respectively perform detection of three sizes, namely large, medium and small; for the detection of small objects, adding an upper sampling layer, setting an anchor point, adding a detection probe, dividing the image into 104 × 104, and detecting the smaller objects; meanwhile, a monocular vision estimation method is adopted to assist the three-dimensional measurement.
Compared with the prior art, the invention has the beneficial effects that:
the improved scheme is used for detecting smaller articles, meanwhile, the rapid measurement is facilitated, and the accuracy is improved.
Secondly, the grabbing robot target positioning system with higher precision and certain robustness is realized.
Drawings
For ease of illustration, the invention is described in detail by the following detailed description and the accompanying drawings.
FIG. 1 is a flow chart of a method for detecting an object in the background art;
FIG. 2 is a flow chart of a three-dimensional measurement method in the background art;
fig. 3 is a schematic structural diagram of the present invention.
Detailed Description
In order that the objects, aspects and advantages of the invention will become more apparent, the invention will be described by way of example only, and with reference to the accompanying drawings. It is to be understood that such description is merely illustrative and not intended to limit the scope of the present invention. The structures, proportions, and dimensions shown in the drawings and described in the specification are only for the purpose of understanding and reading the present disclosure, and are not intended to limit the scope of the present disclosure, which is defined in the claims, and therefore, the present disclosure is not limited to the essential meanings of the technology, and any modifications of the structures, changes of the proportions, or adjustments of the dimensions, should be within the scope of the present disclosure without affecting the efficacy and attainment of the same. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.
It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and/or processing steps closely related to the scheme according to the present invention are shown in the drawings, and other details not so relevant to the present invention are omitted.
The specific implementation mode adopts the following technical scheme:
firstly, a target positioning system is a system combining a target detection method and a three-dimensional measurement method, and the following scheme is designed for designing the system.
1. A preliminary target positioning system is designed according to the existing target detection method and three-dimensional measurement method, and the system is shown in fig. 3.
2. After the preliminary system is completed, the system is modified for the problems existing in the conventional target detection algorithm and the three-dimensional measurement algorithm. The performance of the grabbing robot is improved, and the whole system has certain robustness.
3. After the system is designed, subsequent check is carried out to check the performance of the system, and the problems existing in the system are further improved.
Secondly, researching a target detection algorithm:
the target detection algorithm at the present stage is mainly based on deep learning, and can be basically divided into a one-stage target detection algorithm and a two-stage target detection algorithm, wherein the accuracy of the two-stage target detection algorithm is generally stronger than that of the one-stage target detection algorithm. However, the real-time performance of the two-stage algorithm is inferior to that of the one-stage algorithm, and the working efficiency of the grabbing robot is inevitably influenced if the real-time performance of the target detection algorithm is insufficient in consideration of the working efficiency of the grabbing robot.
2.1, a Yolo target detection algorithm:
yolo is a very representative algorithm in single-stage target detection algorithms, and has been developed to the 5 th generation, which adopts CSPDarknet53 after the 3 rd generation in the backbone network, which is a network framework developed on the residual error network; the 1 × 1 convolution kernel is introduced, and by using the convolution kernel together with the conventional 3 × 3 convolution kernel, the parameter amount is reduced by nearly half compared with the conventional convolution kernel only using 3 × 3, so that the Yolo network has extremely high speed. Meanwhile, the Yolo network has higher accuracy and is suitable for being used as a neural network for grabbing a robot vision system.
On one hand, the Yolo network has a poor detection effect on small objects. On the other hand, the Yolo network sets anchor points in advance, and then regresses the offset of the anchor points and the width and height of the object through the neural network to determine the identification frame of the object. However, the frame thus identified can only identify the approximate range of the object, and cannot identify which point in the frame is on the object. Therefore, when facing an object with a large length-width ratio, there is a large background in the frame when the object is tilted, and the center of the frame is not necessarily on the object. Target positioning is trapped.
2.2, an improved scheme:
the Yolov5 network image enters into prediction after passing through the backhaul, at this time, the network will perform up-sampling, and the image is decomposed into 13 × 13, 26 × 26, 52 × 52 parts in turn, and three detection heads respectively perform detection of three sizes, namely large, medium and small. Therefore, for the detection of small objects, an upper sampling layer can be added, an anchor point is arranged, a detection probe is added, the image is divided into 104 x 104, and the detection of the smaller objects is carried out.
Aiming at the improvement scheme that the above-mentioned provided Yolo detection box is difficult to determine which points are problems on the object, a rotation angle is added to the content of the original data set. And modifying the network to change the original n +1+4(n: the number of types of objects, 1: whether the objects exist at the position, 4: xywh, xy is a correction value of an anchor point, wh is the width and the height of the objects) of the image output from the backbone into n +1+4+1 after upsampling, and adding a regression term of a rotation angle to enable the detection frame to rotate along with the rotation of the objects.
In addition, a super-resolution reconstruction technology utilizing multi-frame fused images can be used, and the super-resolution reconstruction technology is started when the visual system cannot detect the object and has poor detection effect, so as to assist the detection of the images.
Thirdly, the robustness problem of the system:
the key of three-dimensional measurement is the acquisition of depth information, and when the system cannot acquire the depth information or acquires wrong depth information, the measured three-dimensional information is wrong. Therefore, it is necessary to add an auxiliary algorithm to improve the robustness of the system. The subject aims to adopt a monocular vision estimation method to assist three-dimensional measurement, and the purpose of improving the robustness of the system is achieved.
And fourthly, research of three-dimensional measurement:
in the traditional three-dimensional measurement method, binocular stereo vision has poor performance in an environment lacking texture, and mismatching and the like can occur in an environment with periodic texture. Structured light is very sensitive to light and is not as sensitive as binocular stereo vision in resolution, and as distance increases, accuracy also decreases. In order to solve the above problems, binocular stereo IR depth ranging is studied. This kind of camera possesses two IR depth sensor, can use speckle structure light to assist like this when utilizing parallel binocular stereovision to measure the degree of depth, has utilized speckle structure light to mark for the full field of vision, and this makes each facula all unique in the field of vision, so does not have and can't find the characteristic point, perhaps because the mismatch that a large amount of repeated textures caused, very big promotion the matching effect, and then increased substantially the acquisition effect of Z axle degree of depth information. But the baseline of the camera can also be made very small, only 50 mm. The three-dimensional measurement method is suitable for being used as a three-dimensional measurement scheme of the grabbing robot.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (1)
1. The utility model provides a snatch robot target positioning system based on deep learning which characterized in that: the system comprises an image acquisition module, a data processing module and a control module; the image acquisition module is connected with the data processing module through a USB3.0 data line, the data processing module is connected with the control module through a TCP protocol, a target detection algorithm is adopted as a Yolo target detection algorithm, a Yolov5 network image enters prediction after passing through a backhaul, at the moment, the network performs up-sampling, the image is sequentially decomposed into 13 × 13, 26 × 26 and 52 × 52 parts, and three detection heads respectively perform detection of three sizes, namely large, medium and small; for the detection of small objects, adding an upper sampling layer, setting an anchor point, adding a detection probe, dividing the image into 104 × 104, and detecting the smaller objects; meanwhile, a monocular vision estimation method is adopted to assist the three-dimensional measurement.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210460922.5A CN114723827A (en) | 2022-04-28 | 2022-04-28 | Grabbing robot target positioning system based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210460922.5A CN114723827A (en) | 2022-04-28 | 2022-04-28 | Grabbing robot target positioning system based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114723827A true CN114723827A (en) | 2022-07-08 |
Family
ID=82246106
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210460922.5A Pending CN114723827A (en) | 2022-04-28 | 2022-04-28 | Grabbing robot target positioning system based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114723827A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111080693A (en) * | 2019-11-22 | 2020-04-28 | 天津大学 | Robot autonomous classification grabbing method based on YOLOv3 |
| CN113436239A (en) * | 2021-05-18 | 2021-09-24 | 中国地质大学（武汉） | Monocular image three-dimensional target detection method based on depth information estimation |
| KR20220004607A (en) * | 2020-12-25 | 2022-01-11 | 아폴로 인텔리전트 커넥티비티 (베이징) 테크놀로지 씨오., 엘티디. | Target detection method, electronic device, roadside device and cloud control platform |
| CN114255443A (en) * | 2021-12-10 | 2022-03-29 | 深圳市旗扬特种装备技术工程有限公司 | Monocular positioning method, device, equipment and storage medium for traffic vehicle |
-
2022
- 2022-04-28 CN CN202210460922.5A patent/CN114723827A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111080693A (en) * | 2019-11-22 | 2020-04-28 | 天津大学 | Robot autonomous classification grabbing method based on YOLOv3 |
| KR20220004607A (en) * | 2020-12-25 | 2022-01-11 | 아폴로 인텔리전트 커넥티비티 (베이징) 테크놀로지 씨오., 엘티디. | Target detection method, electronic device, roadside device and cloud control platform |
| CN113436239A (en) * | 2021-05-18 | 2021-09-24 | 中国地质大学（武汉） | Monocular image three-dimensional target detection method based on depth information estimation |
| CN114255443A (en) * | 2021-12-10 | 2022-03-29 | 深圳市旗扬特种装备技术工程有限公司 | Monocular positioning method, device, equipment and storage medium for traffic vehicle |
Non-Patent Citations (1)
| Title |
|---|
| S.ZHENKAI: ""yolov3改进4层特征检测层"", pages 1 - 4, Retrieved from the Internet <URL:https://blog.csdn.net/weixin_44076342/article/details/106547312> * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110335295B (en) | Plant point cloud acquisition registration and optimization method based on TOF camera | |
| CN102032875B (en) | Image-processing-based cable sheath thickness measuring method | |
| CN102901444B (en) | Method for detecting component size based on matching pursuit (MP) wavelet filtering and detecting system thereof | |
| CN102221331B (en) | Measuring method based on asymmetric binocular stereovision technology | |
| CN103175485A (en) | Method for visually calibrating aircraft turbine engine blade repair robot | |
| CN103278090A (en) | Visual measurement method for volume of irregular object | |
| CN109580630A (en) | A kind of visible detection method of component of machine defect | |
| Xia et al. | An accurate and robust method for the measurement of circular holes based on binocular vision | |
| He et al. | A critical review for machining positioning based on computer vision | |
| CN113744351A (en) | Underwater structured light measurement calibration method and system based on multi-medium refraction imaging | |
| Xia et al. | Workpieces sorting system based on industrial robot of machine vision | |
| CN115272616A (en) | Indoor scene three-dimensional reconstruction method, system, device and storage medium | |
| CN103308000A (en) | Method for measuring curve object on basis of binocular vision | |
| CN113538378A (en) | Bearing size online detection system based on deep learning | |
| CN110864671B (en) | Robot repeated positioning precision measuring method based on line structured light fitting plane | |
| CN102679937A (en) | Ship steel plate dynamic three-dimension measurement method based on multi-camera vision | |
| CN114092411A (en) | Efficient and rapid binocular 3D point cloud welding spot defect detection method | |
| CN111598172B (en) | Dynamic target grabbing gesture rapid detection method based on heterogeneous depth network fusion | |
| Li et al. | Using laser measuring and SFM algorithm for fast 3D reconstruction of objects | |
| CN115719363B (en) | Environment sensing method and system capable of performing two-dimensional dynamic detection and three-dimensional reconstruction | |
| CN114723827A (en) | Grabbing robot target positioning system based on deep learning | |
| CN116482627A (en) | Combined calibration method based on millimeter wave radar and monocular camera | |
| Sun et al. | Precision work-piece detection and measurement combining top-down and bottom-up saliency | |
| CN110084887B (en) | Three-dimensional reconstruction method for space non-cooperative target relative navigation model | |
| CN114092396A (en) | Method and device for detecting corner collision flaw of packaging box |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |