CN109035322A - A kind of detection of obstacles and recognition methods based on binocular vision - Google Patents
A kind of detection of obstacles and recognition methods based on binocular vision Download PDFInfo
- Publication number
- CN109035322A CN109035322A CN201810783911.4A CN201810783911A CN109035322A CN 109035322 A CN109035322 A CN 109035322A CN 201810783911 A CN201810783911 A CN 201810783911A CN 109035322 A CN109035322 A CN 109035322A
- Authority
- CN
- China
- Prior art keywords
- depth
- camera
- barrier
- binocular
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20228—Disparity calculation for image-based rendering
Abstract
The invention discloses a kind of detection of obstacles and recognition methods based on binocular vision, comprising steps of 1) carrying out calibration 2 to binocular camera) generate left camera depth image；3) picture size of the left camera of the binocular camera acquired is cut to 416*416, then the image after reduction is inputted into YOLOv3 network model, to obtain barrier window coordinate information in the picture and classification information；4) distance of the depth value of the center of barrier window as barrier in the depth image for taking step 3) to obtain, thus the range information of acquired disturbance object.The present invention is based on the detection of obstacles of binocular vision and recognition methods, its can acquired disturbance object range information, also it is capable of the classification information of acquired disturbance object, so as to provide better environment sensing ability for autonomous robot and unmanned automobile, is conducive to it and formulates more accurate decision.
Description
Technical field
The present invention relates to Visual identification technology field, in particular to a kind of detection of obstacles and identification based on binocular vision
Method.
Background technique
The detection and identification of barrier are the key technologies of robot autonomous movement and unmanned automatic driving.Mesh
The method of preceding detection of obstacles has: 1, detection method based on ultrasound；2, based on the detection method of radar；3, it is based on structure light
Detection method；4, vision-based inspection method.But these current methods are the range information for having got barrier, and
It not can determine that the classification information of barrier, and the classification of barrier is also the weight of the formulation decision of autonomous robot and unmanned automobile
Want reference factor.
Summary of the invention
In view of this, a kind of detection of obstacles and recognition methods based on binocular vision of the purpose of the present invention, realization can
The range information of acquired disturbance object is also capable of the classification information of acquired disturbance object, to be autonomous robot and unmanned automobile
Decision-making provides better environment sensing ability.
The present invention is based on the detection of obstacles of binocular vision and recognition methods, comprising the following steps:
1) binocular camera is demarcated:
Left and right camera is demarcated respectively first, obtains the respective internal reference matrix of two cameras and distortion factor
Then matrix obtains eigenmatrix, basis matrix, spin matrix and translation matrix using the three-dimensional correction algorithm in OpenCV；
2) left camera depth image is generated:
First obtain left camera image and right camera image through calibrated binocular camera；Pass through SGBM solid again
Matching algorithm obtains the disparity map of left camera, detects the hole region in disparity map；It is regarded again with hole region is reliable nearby
The mean value of difference is filled perforated, to obtain complete disparity map；It is closed further according to the geometry of parallel binocular vision
System, obtains the conversion formula of following parallax and depth:
Depth=(f*Tx)/disp
In above formula, depth indicates depth map；F indicates normalized focal length；Tx is left camera optical center and right camera light
The distance between heart, referred to as parallax range；Disp is parallax value；A left side is calculated finally by the conversion formula of parallax and depth to take the photograph
As the depth image of head；
3) picture size of the left camera of the binocular camera acquired is cut to 416*416, then will be after reduction
Image inputs YOLOv3 network model, to obtain barrier window coordinate information in the picture and classification information；This step
The window coordinate information of barrier in the window coordinate information i.e. step 2) depth image of obtained barrier in the picture；
4) in the depth image for taking step 3) to obtain the depth value of the center of barrier window as barrier away from
From thus the range information of acquired disturbance object.
Beneficial effects of the present invention:
The present invention is based on the detection of obstacles of binocular vision and recognition methods, can acquired disturbance object range information,
Also it is capable of the classification information of acquired disturbance object, so as to provide better environment sensing energy for autonomous robot and unmanned automobile
Power is conducive to it and formulates more accurate decision.
Detailed description of the invention
Fig. 1 is the flow chart the present invention is based on the detection of obstacles of binocular vision and recognition methods.
Specific embodiment
The invention will be further described with reference to the accompanying drawings and examples.
Detection of obstacles and recognition methods of the present embodiment based on binocular vision, comprising the following steps:
1) binocular camera is demarcated:
Left and right camera is demarcated respectively first, obtains the respective internal reference matrix of two cameras and distortion factor
Then matrix obtains eigenmatrix, basis matrix, spin matrix and translation matrix using the three-dimensional correction algorithm in OpenCV.
2) left camera depth image is generated:
First obtain left camera image and right camera image through calibrated binocular camera；Pass through SGBM solid again
Matching algorithm obtains the disparity map of left camera, detects the hole region in disparity map；It is regarded again with hole region is reliable nearby
The mean value of difference is filled perforated, to obtain complete disparity map；It is closed further according to the geometry of parallel binocular vision
System, obtains the conversion formula of following parallax and depth:
Depth=(f*Tx)/disp
In above formula, depth indicates depth map；F indicates normalized focal length；Tx is left camera optical center and right camera light
The distance between heart, referred to as parallax range；Disp is parallax value；A left side is calculated finally by the conversion formula of parallax and depth to take the photograph
As the depth image of head.
3) picture size of the left camera of the binocular camera acquired is cut to 416*416, then will be after reduction
Image inputs YOLOv3 network model, to obtain barrier window coordinate information in the picture and classification information；This step
The window coordinate information of barrier in the window coordinate information i.e. step 2) depth image of obtained barrier in the picture.
The full name of YOLO are as follows: You Only Look Once:Unified, Real-Time Object Detection,
It is the object detection system based on single Neural that Joseph Redmon and Ali Farhadi et al. was proposed in 2015,
YOLOv3 is its third generation product.The convolutional neural networks model that YOLOv3 network model uses is Darknet-53, it includes
The size of 53 convolutional layers, convolution kernel is 3*3 and two kinds of 1*1, is trained based on COCO data set to network model, works as iteration
Training terminates when number is greater than given number of iterations.
4) in the depth image for taking step 3) to obtain the depth value of the center of barrier window as barrier away from
From thus the range information of acquired disturbance object.
Detection of obstacles and recognition methods in the present embodiment based on binocular vision, can acquired disturbance object distance letter
Breath, is capable of the classification information of acquired disturbance object, also so as to provide better environment sensing for autonomous robot and unmanned automobile
Ability is conducive to it and formulates more accurate decision, for example, when identify to barrier is abiotic stone when, autonomous robot
The decision of detour can be made with unmanned automobile；When identifying barrier is people or other animals, autonomous robot and unmanned vapour
When vehicle can make decision forward again after waiting human or animal leaves.
Finally, it is stated that the above examples are only used to illustrate the technical scheme of the present invention and are not limiting, although referring to compared with
Good embodiment describes the invention in detail, those skilled in the art should understand that, it can be to skill of the invention
Art scheme is modified or replaced equivalently, and without departing from the objective and range of technical solution of the present invention, should all be covered at this
In the scope of the claims of invention.
Claims (1)
1. a kind of detection of obstacles and recognition methods based on binocular vision, it is characterised in that: the following steps are included:
1) binocular camera is demarcated:
Left and right camera is demarcated respectively first, obtains the respective internal reference matrix of two cameras and distortion factor matrix,
Then eigenmatrix, basis matrix, spin matrix and translation matrix are obtained using the three-dimensional correction algorithm in OpenCV；
2) left camera depth image is generated:
First obtain left camera image and right camera image through calibrated binocular camera；Pass through SGBM Stereo matching again
Algorithm obtains the disparity map of left camera, detects the hole region in disparity map；Again with parallax value reliable near hole region
Mean value perforated is filled, to obtain complete disparity map；Further according to the geometrical relationship of parallel binocular vision, obtain
To the conversion formula of following parallax and depth:
Depth=(f*Tx)/disp
In above formula, depth indicates depth map；F indicates normalized focal length；Tx be left camera optical center and right camera optical center it
Between distance, referred to as parallax range；Disp is parallax value；Left camera is calculated finally by the conversion formula of parallax and depth
Depth image；
3) picture size of the left photography/videography head of the binocular camera acquired is cut to 416*416, then by the figure after reduction
As input YOLOv3 network model, to obtain barrier window coordinate information in the picture and classification information；This step obtains
The window coordinate information of barrier in the window coordinate information i.e. step 2) depth image of the barrier arrived in the picture；
4) distance of the depth value of the center of barrier window as barrier in the depth image for taking step 3) to obtain, from
And the range information of acquired disturbance object.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810783911.4A CN109035322A (en) | 2018-07-17 | 2018-07-17 | A kind of detection of obstacles and recognition methods based on binocular vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810783911.4A CN109035322A (en) | 2018-07-17 | 2018-07-17 | A kind of detection of obstacles and recognition methods based on binocular vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109035322A true CN109035322A (en) | 2018-12-18 |
Family
ID=64643603
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810783911.4A Pending CN109035322A (en) | 2018-07-17 | 2018-07-17 | A kind of detection of obstacles and recognition methods based on binocular vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109035322A (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109741405A (en) * | 2019-01-21 | 2019-05-10 | 同济大学 | A kind of depth information acquisition system based on dual structure light RGB-D camera |
| CN109828578A (en) * | 2019-02-22 | 2019-05-31 | 南京天创电子技术有限公司 | A kind of instrument crusing robot optimal route planing method based on YOLOv3 |
| CN109934108A (en) * | 2019-01-31 | 2019-06-25 | 华南师范大学 | The vehicle detection and range-measurement system and implementation method of a kind of multiple target multiple types |
| CN110109457A (en) * | 2019-04-29 | 2019-08-09 | 北方民族大学 | A kind of intelligent sound blind-guidance robot control method and control system |
| CN110132243A (en) * | 2019-05-31 | 2019-08-16 | 南昌航空大学 | A kind of modularization positioning system based on deep learning and ranging |
| CN110222557A (en) * | 2019-04-22 | 2019-09-10 | 北京旷视科技有限公司 | Real-time detection method, device, system and the storage medium of road conditions |
| CN110298330A (en) * | 2019-07-05 | 2019-10-01 | 东北大学 | A kind of detection of transmission line polling robot monocular and localization method |
| CN110398226A (en) * | 2019-05-21 | 2019-11-01 | 重庆邮电大学 | A kind of monocular vision distance measuring method for advanced DAS (Driver Assistant System) |
| CN111239684A (en) * | 2020-01-17 | 2020-06-05 | 中航华东光电（上海）有限公司 | Binocular fast distance measurement method based on YoloV3 deep learning |
| CN111551920A (en) * | 2020-04-16 | 2020-08-18 | 重庆大学 | Three-dimensional target real-time measurement system and method based on target detection and binocular matching |
| CN111611918A (en) * | 2020-05-20 | 2020-09-01 | 重庆大学 | Traffic flow data set acquisition and construction method based on aerial photography data and deep learning |
| CN111724432A (en) * | 2020-06-04 | 2020-09-29 | 杭州飞步科技有限公司 | Object three-dimensional detection method and device |
| CN111950428A (en) * | 2020-08-06 | 2020-11-17 | 东软睿驰汽车技术（沈阳）有限公司 | Target obstacle identification method and device and carrier |
| CN112418040A (en) * | 2020-11-16 | 2021-02-26 | 南京邮电大学 | Binocular vision-based method for detecting and identifying fire fighting passage occupied by barrier |
| CN114046796A (en) * | 2021-11-04 | 2022-02-15 | 南京理工大学 | Intelligent wheelchair autonomous walking algorithm, device and medium |
| CN114608522A (en) * | 2022-03-21 | 2022-06-10 | 沈阳理工大学 | Vision-based obstacle identification and distance measurement method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103714532A (en) * | 2013-12-09 | 2014-04-09 | 广西科技大学 | Method for automatically detecting obstacles based on binocular vision |
| CN106156748A (en) * | 2016-07-22 | 2016-11-23 | 浙江零跑科技有限公司 | Traffic scene participant's recognition methods based on vehicle-mounted binocular camera |
| CN107636679A (en) * | 2016-12-30 | 2018-01-26 | 深圳前海达闼云端智能科技有限公司 | A kind of obstacle detection method and device |
| WO2018023333A1 (en) * | 2016-08-01 | 2018-02-08 | SZ DJI Technology Co., Ltd. | System and method for obstacle avoidance |
-
2018
- 2018-07-17 CN CN201810783911.4A patent/CN109035322A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103714532A (en) * | 2013-12-09 | 2014-04-09 | 广西科技大学 | Method for automatically detecting obstacles based on binocular vision |
| CN106156748A (en) * | 2016-07-22 | 2016-11-23 | 浙江零跑科技有限公司 | Traffic scene participant's recognition methods based on vehicle-mounted binocular camera |
| WO2018023333A1 (en) * | 2016-08-01 | 2018-02-08 | SZ DJI Technology Co., Ltd. | System and method for obstacle avoidance |
| CN107636679A (en) * | 2016-12-30 | 2018-01-26 | 深圳前海达闼云端智能科技有限公司 | A kind of obstacle detection method and device |
Non-Patent Citations (1)
| Title |
|---|
| 冯瑾: "基于双目立体视觉的移动机器人障碍物检测技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109741405A (en) * | 2019-01-21 | 2019-05-10 | 同济大学 | A kind of depth information acquisition system based on dual structure light RGB-D camera |
| CN109741405B (en) * | 2019-01-21 | 2021-02-02 | 同济大学 | Depth information acquisition system based on dual structured light RGB-D camera |
| CN109934108A (en) * | 2019-01-31 | 2019-06-25 | 华南师范大学 | The vehicle detection and range-measurement system and implementation method of a kind of multiple target multiple types |
| CN109934108B (en) * | 2019-01-31 | 2023-01-31 | 华南师范大学 | Multi-target and multi-type vehicle detection and distance measurement system and implementation method |
| CN109828578A (en) * | 2019-02-22 | 2019-05-31 | 南京天创电子技术有限公司 | A kind of instrument crusing robot optimal route planing method based on YOLOv3 |
| CN109828578B (en) * | 2019-02-22 | 2020-06-16 | 南京天创电子技术有限公司 | Instrument inspection robot optimal route planning method based on YOLOv3 |
| CN110222557B (en) * | 2019-04-22 | 2021-09-21 | 北京旷视科技有限公司 | Real-time road condition detection method, device and system and storage medium |
| CN110222557A (en) * | 2019-04-22 | 2019-09-10 | 北京旷视科技有限公司 | Real-time detection method, device, system and the storage medium of road conditions |
| CN110109457A (en) * | 2019-04-29 | 2019-08-09 | 北方民族大学 | A kind of intelligent sound blind-guidance robot control method and control system |
| CN110398226A (en) * | 2019-05-21 | 2019-11-01 | 重庆邮电大学 | A kind of monocular vision distance measuring method for advanced DAS (Driver Assistant System) |
| CN110132243A (en) * | 2019-05-31 | 2019-08-16 | 南昌航空大学 | A kind of modularization positioning system based on deep learning and ranging |
| CN110298330A (en) * | 2019-07-05 | 2019-10-01 | 东北大学 | A kind of detection of transmission line polling robot monocular and localization method |
| CN111239684A (en) * | 2020-01-17 | 2020-06-05 | 中航华东光电（上海）有限公司 | Binocular fast distance measurement method based on YoloV3 deep learning |
| CN111551920A (en) * | 2020-04-16 | 2020-08-18 | 重庆大学 | Three-dimensional target real-time measurement system and method based on target detection and binocular matching |
| CN111611918A (en) * | 2020-05-20 | 2020-09-01 | 重庆大学 | Traffic flow data set acquisition and construction method based on aerial photography data and deep learning |
| CN111611918B (en) * | 2020-05-20 | 2023-07-21 | 重庆大学 | Traffic flow data set acquisition and construction method based on aerial data and deep learning |
| CN111724432A (en) * | 2020-06-04 | 2020-09-29 | 杭州飞步科技有限公司 | Object three-dimensional detection method and device |
| CN111724432B (en) * | 2020-06-04 | 2023-08-22 | 杭州飞步科技有限公司 | Object three-dimensional detection method and device |
| CN111950428A (en) * | 2020-08-06 | 2020-11-17 | 东软睿驰汽车技术（沈阳）有限公司 | Target obstacle identification method and device and carrier |
| CN112418040B (en) * | 2020-11-16 | 2022-08-26 | 南京邮电大学 | Binocular vision-based method for detecting and identifying fire fighting passage occupied by barrier |
| CN112418040A (en) * | 2020-11-16 | 2021-02-26 | 南京邮电大学 | Binocular vision-based method for detecting and identifying fire fighting passage occupied by barrier |
| CN114046796A (en) * | 2021-11-04 | 2022-02-15 | 南京理工大学 | Intelligent wheelchair autonomous walking algorithm, device and medium |
| CN114608522A (en) * | 2022-03-21 | 2022-06-10 | 沈阳理工大学 | Vision-based obstacle identification and distance measurement method |
| CN114608522B (en) * | 2022-03-21 | 2023-09-26 | 沈阳理工大学 | Obstacle recognition and distance measurement method based on vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109035322A (en) | A kind of detection of obstacles and recognition methods based on binocular vision | |
| JP6288221B2 (en) | Enhanced layer-based object detection by deep convolutional neural networks | |
| CA3087536C (en) | Fish biomass, shape, and size determination | |
| WO2022188379A1 (en) | Artificial intelligence system and method serving electric power robot | |
| CN112132972B (en) | Three-dimensional reconstruction method and system for fusing laser and image data | |
| CN105313782B (en) | Vehicle travel assist system and its method | |
| CN108053449A (en) | Three-dimensional rebuilding method, device and the binocular vision system of binocular vision system | |
| CN110147706A (en) | The recognition methods of barrier and device, storage medium, electronic device | |
| CN109048926A (en) | A kind of intelligent robot obstacle avoidance system and method based on stereoscopic vision | |
| CN107491071B (en) | Intelligent multi-robot cooperative mapping system and method thereof | |
| US11100669B1 (en) | Multimodal three-dimensional object detection | |
| CN106444837A (en) | Obstacle avoiding method and obstacle avoiding system for unmanned aerial vehicle | |
| CN106156748A (en) | Traffic scene participant's recognition methods based on vehicle-mounted binocular camera | |
| CN110673632A (en) | Unmanned aerial vehicle autonomous obstacle avoidance method and device based on visual SLAM | |
| CN116258817B (en) | Automatic driving digital twin scene construction method and system based on multi-view three-dimensional reconstruction | |
| CN110298330B (en) | Monocular detection and positioning method for power transmission line inspection robot | |
| CN108985193A (en) | A kind of unmanned plane portrait alignment methods based on image detection | |
| CN113408584A (en) | RGB-D multi-modal feature fusion 3D target detection method | |
| CN111209840A (en) | 3D target detection method based on multi-sensor data fusion | |
| Ozcan et al. | A novel fusion method with thermal and RGB-D sensor data for human detection | |
| CN113681552B (en) | Five-dimensional grabbing method for robot hybrid object based on cascade neural network | |
| WO2022061632A1 (en) | Obstacle detection method and apparatus, and unmanned aerial vehicle and storage medium | |
| NO20201081A1 (en) | Generating three dimensional skeleton representations of aquatic animals using machine learning | |
| CN113110562A (en) | Unmanned aerial vehicle obstacle avoidance device based on multiple wide-angle cameras and obstacle avoidance method thereof | |
| CN115359067A (en) | Continuous convolution network-based point-by-point fusion point cloud semantic segmentation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181218 |