CN104296657A - Stone wall blasthole detecting and positioning device based on binocular vision and positioning method - Google Patents
Stone wall blasthole detecting and positioning device based on binocular vision and positioning method Download PDFInfo
- Publication number
- CN104296657A CN104296657A CN201410535284.4A CN201410535284A CN104296657A CN 104296657 A CN104296657 A CN 104296657A CN 201410535284 A CN201410535284 A CN 201410535284A CN 104296657 A CN104296657 A CN 104296657A
- Authority
- CN
- China
- Prior art keywords
- camera
- hole
- image
- coordinate system
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention provides a stone wall blasthole detecting and positioning device based on binocular vision and a positioning method. The device comprises a main video camera and an auxiliary video camera. The main video camera and the auxiliary video camera are mounted on a multi-joint movable manipulator and connected with an image acquisition card through signal lines. The image acquisition card and the multi-joint movable manipulator are connected with a computer through signal lines. According to the device, the main video camera is used for searching for, partitioning and locking a hole, and meanwhile the auxiliary video camera is used for photographing images and storing the images for standby application; after the target hole is locked through the computer, the image corresponding to the target hole in the auxiliary video camera is called and is combined with a target hole image obtained by the main video camera for binocular calibration, and then the current space coordinate position of the target hole relative to the working manipulator can be positioned.
Description
Technical field
The invention belongs to observation and control technology field, be specifically related to a kind of cliff blast hole detection & localization device based on binocular vision and localization method, namely take same blast hole image by two video cameras simultaneously, through Image processing and compute, obtain the three-dimensional coordinate of target hole.
Background technology
Usually need to carry out explosion to massif in large scale civil engineering, generally first hole on cliff, then load explosive.But, because cliff in field environment is very high, surface area large and surface topography irregular, not only labour intensity is large, efficiency is low to adopt Artificial Intervention explosive, and danger is very high, explosive is implanted according to automation equipment, again because the operator on most of hole and equipment makes the frequent neglected loading of equipment operator or the many holes of misloading apart from the reason such as too far away, thus the automatic detection & localization device in exploitation blasting hole is needed, for operator provides empty position information, improve the accuracy that explosive is implanted, reduce the labour intensity of neglected loading rate and operator.
Summary of the invention
The invention provides a kind of can replace manually automatically completing the detection & localization of cliff blast hole, based on the cliff blast hole detection & localization device of binocular vision and localization method.
For achieving the above object, the present invention adopts following technical solution: a kind of cliff blast hole detection & localization device based on binocular vision, comprise main camera and auxiliary camera, main camera and auxiliary camera are arranged on multi-joint movable mechanical on hand, main camera is connected with image pick-up card by signal wire with auxiliary camera, and image pick-up card and multi-joint movable mechanical hand are connected with computing machine by signal wire.
Described controller inbound pacing sensor and the signal of displacement transducer are also imported into computing machine.
The method utilizing said apparatus to locate cliff blast hole, comprises the following steps:
1) blast hole search, detection: take from different orientation and visual angle the region having blast hole to distribute respectively by main camera and auxiliary camera simultaneously, and by image pick-up card by the picture transfer of shooting to vision module, vision module detects the blast hole in picture and locks one of them hole is current goal hole;
2) target hole three-dimensional coordinate is determined: processed by the image of the image processing software in vision module to hole, obtains the two dimensional image coordinate P of target hole in image captured by main camera and auxiliary camera respectively
1(u, v) and P
2(u ', v ').Now, the optical axis of main camera is parallel to the optical axis of auxiliary camera, obtains the three-dimensional coordinate (X, Y, Z) of target hole in the coordinate system C being reference with main camera (1) by triangle relation;
In formula (1), f is camera focus, and L is the distance between two video cameras, and d=u-u ' is parallax, (u
0, v
0) be the coordinate of camera imaging planar central pixel; Namely the three-dimensional coordinate of target hole center in camera coordinate system C is obtained thus;
Further, camera coordinate system C and mechanical arm base coordinate system O (X is supposed
0, Y
0, Z
0) between space conversion matrix be T, each parameter wherein in T is recorded in real time by the displacement transducer installed in the control module, then camera coordinate system C can be converted in robot coordinate system O:
After this, after obtaining arbitrary target hole, by above calculation procedure, by target hole space coordinate conversion in mechanical arm base coordinate system, realize target hole is relative to the space orientation of robot coordinate system.
The present invention is at blast hole search, detection-phase, and main camera and auxiliary camera are taken simultaneously, but only process main camera image, detect whether porose existence, the Image Saving of auxiliary camera, can reduce calculated amount, improves the real-time of detection system work.
Accompanying drawing explanation
Below in conjunction with drawings and Examples, the invention will be further described.
Fig. 1 is system of the present invention composition schematic diagram.
Fig. 2 is pretreated blast hole image.
Fig. 3 is blast hole binocular calibration illustraton of model, and, f is focal length of camera in Fig. 3 wherein, is the known quantity determined by camera; u
0, v
0it is the coordinate of camera imaging planar central point; P is target hole center; P
1, P
2be respectively the imager coordinate of center, hole P in video camera 1,2; U, u ' be respectively the horizontal ordinate of P imaging in two video cameras; C, C ' be respectively the true origin of two camera coordinate systems.
Embodiment
Be below specific embodiments of the invention and by reference to the accompanying drawings, technical scheme of the present invention is further described, but the present invention be not limited to these embodiments.
Cliff blast hole detection & localization apparatus structure based on binocular vision provided by the invention is as follows: as shown in Figure 1, comprise main camera 1 and auxiliary camera 2, main camera 1 and auxiliary camera 2 are arranged on multi-joint movable mechanical hand 3, main camera 1 is connected with image pick-up card 4 by signal wire with auxiliary camera 2, and image pick-up card 4 and multi-joint movable mechanical hand 3 are connected with computing machine 5 by signal wire.Described controller 7 inbound pacing sensor and the signal of displacement transducer are also imported into computing machine 5.
In the present invention, for reducing calculated amount, improve the real-time of detection system work, at blast hole search, detection-phase, main camera 1 and auxiliary camera 2 are taken simultaneously, but vision module only processes main camera 1 image, detect whether porose existence, auxiliary camera image 2 is preserved.Namely the image only relying on main camera 1 acquisition carries out hole search and judgement.
If computing machine 5 judges in shooting area porose in search procedure, then split in the ken, lock each hole (if with when photographing multiple hole in piece image), and to lock a certain hole be current goal hole implants explosive in order to mechanical arm 3.Call the image that the auxiliary camera 2 corresponding with this image obtains, segmentation, lock onto target hole simultaneously.
Be below an operational instances of the present invention:
After two main cameras 1 being in different azimuth and auxiliary camera 2 take blast hole image simultaneously, through image processing software, the pre-service such as denoising, enhancing is carried out to blast hole image by conventional method, and auto Segmentation is carried out in the hole in image, locking a hole is current goal hole, as hole 2.As shown in Figure 2.
The two-dimensional coordinate of center, hypothetical target hole in two camera reviews is respectively P
1(u, v) and P
2(u ', v ').Now, if two camera parameters (as focal length, resolution etc.) are identical, and the optical axis of two video cameras is parallel, then can obtain the three-dimensional coordinate (X, Y, Z) of target hole in the coordinate system C being reference with video camera 1 by simple triangle relation.
The binocular calibration illustraton of model of the blast hole according to Fig. 3, can obtain:
In formula (1), f is camera focus, and L is the distance between two video cameras.D=u-u ' is parallax, (u
0, v
0) be the coordinate of plane of delineation central pixel point; Namely the three-dimensional coordinate of target hole center in camera coordinate system C is obtained thus;
Further, camera coordinate system C and mechanical arm base coordinate system O (X is supposed
0, Y
0, Z
0) between space conversion matrix be T, each parameter wherein in T is recorded in real time by each displacement transducer installed in the control module, then camera coordinate system C can be converted in robot coordinate system O:
After this, after obtaining arbitrary target hole, by above calculation procedure, by target hole space coordinate conversion in mechanical arm base coordinate system, realize target hole is relative to the space orientation of robot coordinate system.
Be below a located instance of the present invention:
By actual measurement, target hole 2 diameter in shown in known Fig. 2 is 150mm, and its central point is (65.00 at the coordinate of mechanical arm base coordinate system, 780.00,1550.00), unit is millimeter (below unless specifically indicated, unit is millimeter).And the coordinate of the central point of blast hole 2 in image coordinate system is respectively P
1(1339.50,1293.71) (unit: pixel), P
2(831.40,1292.65), now, f=25.5mm, k=0.01mm, the distance L=150mm between two video cameras, (ku
0, kv
0)=(16.32,12.24).Therefore be C (-86.35 ,-20.58,752.80) according to the coordinate of central point in camera coordinate system that formula (1) can be calculated blast hole 2.
Now, camera coordinates is T (150.00,800.00 relative to robot base transition matrix, 800.00), the coordinate of center in robot coordinate system that (2) formula of substitution can obtain blast hole 2 is O (63.65,779.42,1552.80).
To when result:
Contrasted from the actual coordinate at above blast hole 2 center and surving coordinate, maximum error between Two coordinate is 2.8mm (Z-direction), therefore the present invention has higher measuring accuracy (especially for larger-diameter hole) and good realizability.
Claims (4)
1. the cliff blast hole detection & localization device based on binocular vision, it is characterized in that: comprise main camera (1) and auxiliary camera (2), main camera (1) and auxiliary camera (2) are arranged on multi-joint movable mechanical hand (3), main camera (1) is connected with image pick-up card (4) by signal wire with auxiliary camera (2), and image pick-up card (4) and multi-joint movable mechanical hand (3) are connected with computing machine (5) by signal wire and controller (7).
2. a kind of cliff blast hole detection & localization device based on binocular vision according to claim 1, is characterized in that: described controller (7) inbound pacing sensor and the signal of displacement transducer are also imported into computing machine (5).
3. the method utilizing the device in claim 1-2 described in arbitrary claim to locate cliff blast hole, is characterized in that: comprise the following steps:
1) blast hole search, detection: take from different orientation and visual angle the region (6) having blast hole to distribute respectively by main camera (1) and auxiliary camera (2) simultaneously, and passing through image pick-up card (4) by the picture transfer of shooting to vision module, vision module detects the blast hole in picture and locks one of them hole is current goal hole;
2) target hole three-dimensional coordinate is determined: processed by the image of the image processing software in vision module to hole, obtains the two dimensional image coordinate P of target hole in main camera (1) and the captured image of auxiliary camera (2) respectively
1(u, v) and P
2(u ', v ').Now, the optical axis of main camera is parallel to the optical axis of auxiliary camera, obtains the three-dimensional coordinate (X, Y, Z) of target hole in the coordinate system C being reference with main camera (1) by triangle relation;
In formula (1), f is camera focus, and L is the distance between two video cameras, and d=u-u ' is parallax, (u
0, v
0) be the coordinate of camera imaging planar central pixel; Namely the three-dimensional coordinate of target hole center in camera coordinate system C is obtained thus;
Further, camera coordinate system C and mechanical arm base coordinate system O (X is supposed
0, Y
0, Z
0) between space conversion matrix be T, each parameter wherein in T is recorded in real time by the displacement transducer installed in the control module, then camera coordinate system C can be converted in robot coordinate system O:
After this, after obtaining arbitrary target hole, by above calculation procedure, by target hole space coordinate conversion in mechanical arm base coordinate system, realize target hole is relative to the space orientation of robot coordinate system.
4. localization method according to claim 3, it is characterized in that: at blast hole search, detection-phase, main camera (1) and auxiliary camera (2) are taken simultaneously, but only main camera image is processed, detect whether porose existence, the Image Saving of auxiliary camera (2), can reduce calculated amount, improves the real-time of detection system work.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410535284.4A CN104296657B (en) | 2014-10-11 | 2014-10-11 | The detection of a kind of cliff blast hole based on binocular vision and positioner and localization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410535284.4A CN104296657B (en) | 2014-10-11 | 2014-10-11 | The detection of a kind of cliff blast hole based on binocular vision and positioner and localization method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104296657A true CN104296657A (en) | 2015-01-21 |
| CN104296657B CN104296657B (en) | 2016-09-07 |
Family
ID=52316493
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410535284.4A Active CN104296657B (en) | 2014-10-11 | 2014-10-11 | The detection of a kind of cliff blast hole based on binocular vision and positioner and localization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104296657B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105538345A (en) * | 2016-01-27 | 2016-05-04 | 华南理工大学 | Intelligent mechanical arm based on multiple cameras and positioning and assembling method |
| CN111586290A (en) * | 2020-04-07 | 2020-08-25 | 延锋伟世通电子科技（上海）有限公司 | Position calibration method for camera for optical test of vehicle-mounted head-up display |
| CN111664788A (en) * | 2019-03-06 | 2020-09-15 | 东莞中科蓝海智能视觉科技有限公司 | Washing machine inner barrel hole position detection method based on intelligent vision |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102128589A (en) * | 2010-01-20 | 2011-07-20 | 中国科学院自动化研究所 | Method for correcting azimuth errors of inner bore of part in process of assembling axle hole |
| US20130058581A1 (en) * | 2010-06-23 | 2013-03-07 | Beihang University | Microscopic Vision Measurement Method Based On Adaptive Positioning Of Camera Coordinate Frame |
| CN103028816A (en) * | 2012-12-24 | 2013-04-10 | 上海电机学院 | Welding device and method based on binocular vision sensing of structured light |
| JP2013130511A (en) * | 2011-12-22 | 2013-07-04 | Mitsubishi Heavy Ind Ltd | Underwater hole measurement device and underwater reracking method using the device |
| CN204165516U (en) * | 2014-10-11 | 2015-02-18 | 三峡大学 | A kind of cliff blast hole detection & localization device based on binocular vision |
-
2014
- 2014-10-11 CN CN201410535284.4A patent/CN104296657B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102128589A (en) * | 2010-01-20 | 2011-07-20 | 中国科学院自动化研究所 | Method for correcting azimuth errors of inner bore of part in process of assembling axle hole |
| US20130058581A1 (en) * | 2010-06-23 | 2013-03-07 | Beihang University | Microscopic Vision Measurement Method Based On Adaptive Positioning Of Camera Coordinate Frame |
| JP2013130511A (en) * | 2011-12-22 | 2013-07-04 | Mitsubishi Heavy Ind Ltd | Underwater hole measurement device and underwater reracking method using the device |
| CN103028816A (en) * | 2012-12-24 | 2013-04-10 | 上海电机学院 | Welding device and method based on binocular vision sensing of structured light |
| CN204165516U (en) * | 2014-10-11 | 2015-02-18 | 三峡大学 | A kind of cliff blast hole detection & localization device based on binocular vision |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105538345A (en) * | 2016-01-27 | 2016-05-04 | 华南理工大学 | Intelligent mechanical arm based on multiple cameras and positioning and assembling method |
| CN105538345B (en) * | 2016-01-27 | 2017-09-26 | 华南理工大学 | A kind of puma manipulator and positioning assembly method based on many camera lenses |
| US10899014B2 (en) | 2016-01-27 | 2021-01-26 | South China University Of Technology | Multiple lens-based smart mechanical arm and positioning and assembly method thereof |
| CN111664788A (en) * | 2019-03-06 | 2020-09-15 | 东莞中科蓝海智能视觉科技有限公司 | Washing machine inner barrel hole position detection method based on intelligent vision |
| CN111586290A (en) * | 2020-04-07 | 2020-08-25 | 延锋伟世通电子科技（上海）有限公司 | Position calibration method for camera for optical test of vehicle-mounted head-up display |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104296657B (en) | 2016-09-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104280740A (en) | Device for jointly positioning blast hole based on camera and laser distance measuring sensor and positioning method | |
| CN111127568B (en) | Camera pose calibration method based on spatial point location information | |
| CN110246175A (en) | Intelligent Mobile Robot image detecting system and method for the panorama camera in conjunction with holder camera | |
| CN110450129B (en) | Carrying advancing method applied to carrying robot and carrying robot thereof | |
| CN106826817A (en) | Double feedback mechanical arm automatic assembling and disassembling system and methods | |
| EP3771198B1 (en) | Target tracking method and device, movable platform and storage medium | |
| AU2015275035B2 (en) | A method and system to identify a position of a measurement pole | |
| WO2018143263A1 (en) | Photographing control device, photographing control method, and program | |
| CN103020952A (en) | Information processing apparatus and information processing method | |
| CN108214487A (en) | Based on the positioning of the robot target of binocular vision and laser radar and grasping means | |
| CN104217441A (en) | Mechanical arm positioning fetching method based on machine vision | |
| CN110136047B (en) | Method for acquiring three-dimensional information of static target in vehicle-mounted monocular image | |
| CN104002602A (en) | Laser activation device with machining precision correcting function and laser activation method | |
| CN204165516U (en) | A kind of cliff blast hole detection & localization device based on binocular vision | |
| CN204086539U (en) | A kind of device based on video camera and laser range sensor co-located blast hole | |
| CN104296658B (en) | The detection of a kind of cliff blast hole based on virtual binocular vision and positioner and localization method | |
| CN104296657A (en) | Stone wall blasthole detecting and positioning device based on binocular vision and positioning method | |
| CN111213159A (en) | Image processing method, device and system | |
| CN110355758B (en) | Machine following method and equipment and following robot system | |
| CN103707300A (en) | Manipulator device | |
| CN113280209B (en) | System for detecting pipeline excess, use method of system and detection method | |
| Muffert et al. | The estimation of spatial positions by using an omnidirectional camera system | |
| CN110080326A (en) | A kind of discharge method, controller, excavator, electronic equipment and storage medium | |
| CN113701750A (en) | Fusion positioning system of underground multi-sensor | |
| CN204202556U (en) | A kind of cliff blast hole detection & localization device based on virtual binocular vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20211110 Address after: 276000 No. 11, Mengshan fifth road, economic development zone, Mengyin County, Linyi City, Shandong Province Patentee after: SHANDONG YUEHAO AUTOMATION EQUIPMENT Co.,Ltd. Address before: 202150 room 35187, building 3, No. 1800, Panyuan Road, Changxing Town, Chongming District, Shanghai (Shanghai Taihe Economic Development Zone) Patentee before: Shanghai Oubei Industrial Co.,Ltd. Effective date of registration: 20211110 Address after: 202150 room 35187, building 3, No. 1800, Panyuan Road, Changxing Town, Chongming District, Shanghai (Shanghai Taihe Economic Development Zone) Patentee after: Shanghai Oubei Industrial Co.,Ltd. Address before: 443002 No. 8, University Road, Yichang City, Hubei Province Patentee before: CHINA THREE GORGES University |
|
| TR01 | Transfer of patent right |